Surgical robot platform

ABSTRACT

A medical robot system, including a robot coupled to an effectuator element with the robot configured for controlled movement and positioning. The system may include a transmitter configured to emit one or more signals, and the transmitter is coupled to an instrument coupled to the effectuator element. The system may further include a motor assembly coupled to the robot and a plurality of receivers configured to receive the one or more signals emitted by the transmitter. A control unit is coupled to the motor assembly and the plurality of receivers, and the control unit is configured to supply one or more instruction signals to the motor assembly. The instruction signals can be configured to cause the motor assembly to selectively move the effectuator element and is further configured to (i) calculate a position of the at least one transmitter by analysis of the signals received by the plurality of receivers; (ii) display the position of the at least one transmitter with respect to the body of the patient; and (iii) selectively control actuation of the motor assembly in response to the signals received by the plurality of receivers.

RELATED APPLICATIONS

This application is continuation of U.S. patent application Ser. No.13/924,505 filed on Jun. 21, 2013 which claims priority under 35 U.S.C.§119 to U.S. Provisional Patent Application No. 61/662,702 filed on Jun.21, 2012 and U.S. Provisional Patent Application No. 61/800,527 filed onMar. 15, 2013, which are incorporated herein by reference in theirentirety.

BACKGROUND

Various medical procedures require the precise localization of athree-dimensional position of a surgical instrument within the body inorder to effect optimized treatment. For example, some surgicalprocedures to fuse vertebrae require that a surgeon drill multiple holesinto the bone structure at specific locations. To achieve high levels ofmechanical integrity in the fusing system, and to balance the forcescreated in the bone structure, it is necessary that the holes aredrilled at the correct location. Vertebrae, like most bone structures,have complex shapes made up of non-planar curved surfaces making preciseand perpendicular drilling difficult. Conventionally, a surgeon manuallyholds and positions a drill guide tube by using a guidance system tooverlay the drill tube's position onto a three dimensional image of thebone structure. This manual process is both tedious and time consuming.The success of the surgery is largely dependent upon the dexterity ofthe surgeon who performs it.

Limited robotic assistance for surgical procedures is currentlyavailable. For example, the da Vinci® medical robot system (da Vinci® isa registered trademark of Intuitive Surgical) is a robot used in certainsurgical applications. In the da Vinci® system, the user controlsmanipulators that control a robotic actuator. The system converts thesurgeon's gross movements into micro-movements of the robotic actuator.Although the da Vinci® system eliminates hand tremor and provides theuser with the ability to work through a small opening, like many of therobots commercially available today, it is expensive, obtrusive, and thesetup is cumbersome. Further, for procedures such as thoracolumbarpedicle screw insertion, these conventional methods are known to beerror-prone and tedious.

One of the characteristics of many of the current robots used insurgical applications which make them error prone is that they use anarticular arm based on a series of rotational joints. The use of anarticular system may create difficulties in arriving at an accuratelytargeted location because the level of any error is increased over eachjoint in the articular system.

SUMMARY

Some embodiments of the invention provide a surgical robot (andoptionally an imaging system) that utilizes a Cartesian positioningsystem that allows movement of a surgical instrument to be individuallycontrolled in an x-axis, y-axis and z-axis. In some embodiments, thesurgical robot can include a base, a robot arm coupled to and configuredfor articulation relative to the base, as well as an end-effectuatorcoupled to a distal end of the robot arm. The effectuator element caninclude the surgical instrument or can be configured for operativecoupling to the surgical instrument. Some embodiments of the inventionallow the roll, pitch and yaw rotation of the end-effectuator and/orsurgical instrument to be controlled without creating movement along thex-axis, y-axis, or z-axis.

In some embodiments, the end-effectuator can include a guide tube, atool, and/or a penetrating shaft with a leading edge that is eitherbeveled (shaft cross-cut at an angle) or non-beveled (shaft ending in apointed tip). In some embodiments, a non-beveled end-effectuator elementcan be employed to ablate a pathway through tissue to reach the targetposition while avoiding the mechanical forces and deflection created bya typical bevel tissue cutting system.

Some embodiments of the surgical robot can include a motor assemblycomprising three linear motors that separately control movement of theeffectuator element and/or surgical instrument on the respective x-, y-and z-axes. These separate motors can provide a degree of accuracy thatis not provided by conventional surgical robots, thereby giving thesurgeon the capability of more exactly determining position and strikeangles on a three dimensional image.

In some embodiments, at least one RF transmitter can be mounted on theeffectuator element and/or the surgical instrument. Three or more RFreceivers can be mounted in the vicinity of the surgical robot. Thelocation of the RF transmitter and, therefore, the surgical instrument,can be accurately determined by analyzing the RF signals that areemitted from the RF transmitter. For example, by measuring the time offlight of the RF signal from the transmitter to the RF receivers thatare positioned at known locations, the position of the end-effectuatorelement with respect to a patient can be determined. In someembodiments, a physician or surgeon can perform epidural injections ofsteroids into a patient to alleviate back pain without the use of x-raysas is currently required with x-ray fluoroscopic techniques.

Some embodiments of the invention use RF feedback to actively controlthe movement of the surgical robot. For example, RF signals can be sentby the RF transmitter on an iterative basis and then analyzed in aniterative process to allow the surgical robot to automatically move theeffectuator element and/or surgical instrument to a desired locationwithin a patient's body. The location of the effectuator element and/orsurgical instrument can be dynamically updated and, optionally, can bedisplayed to a user in real-time.

In some embodiments, at least one RF transmitter can be disposed onother elements of the surgical robot, or anywhere within the room wherean invasive procedure is taking place, in order to track other devices.

Some embodiments of the invention dispose one or more RF transmitters onthe anatomical part of the patient that is the target of the invasiveprocedure. This system can be used to correct the movement of thesurgical robot in the event the anatomical target moves during theprocedure.

In some embodiments, the system can be configured to automaticallyposition and rigidly hold the end-effectuator and/or the surgicalinstrument in accurate alignment with a required trajectory, such as,for example, a selected trajectory of a pedicle screw during pediclescrew insertion procedures. In case of movement of the patient, thesystem can be configured to automatically adjust the position of therobot to maintain desired alignment relative to an anatomical region ofinterest.

DESCRIPTION OF THE DRAWINGS

FIG. 1 is a partial perspective view of a room in which a medicalprocedure is taking place by using a surgical robot, the movement ofwhich is controlled by analysis of RF signals that are emitted from aninside the patient and received by RF receivers mounted therein.

FIG. 2 is a perspective view of a surgical robot according to anembodiment of the invention.

FIGS. 3A-3B are perspective views of the surgical robot illustrated inFIG. 2, which show the movement of the base of the surgical robot in thez-axis direction in accordance with an embodiment of the invention.

FIG. 4 is a partial perspective view of the surgical robot of FIG. 2which shows how the robot arm can be moved in the x-axis direction.

FIGS. 5A-5B are partial perspective views of the surgical robot of FIG.2, which show how the robot arm can be moved in the y-axis direction.

FIG. 6 is a perspective view of a portion of the robot arm of FIG. 2showing how an effectuator element can be twisted about a y-axis.

FIG. 7 is a perspective view of a portion of a robot arm of FIG. 2showing how an effectuator element can be pivoted about a pivot axisthat is perpendicular to the y-axis.

FIGS. 8A-8B are partial perspective views of the surgical robot of FIG.2, which show the movement of a surgical instrument 35 along the z-axisfrom an effectuator element.

FIG. 9 is a system diagram which shows local positioning sensors, acontrolling PC, and a Radiofrequency (RF) transmitter in accordance withan embodiment of the invention.

FIG. 10 is a system diagram of the controlling PC, user input, andmotors for controlling the robot in accordance with an embodiment of theinvention.

FIG. 11 is a flow chart diagram for general operation of a surgicalrobot in accordance with one embodiment of the invention.

FIG. 12 is a flow chart diagram for a closed screw/needle insertionperformed using a surgical robot in accordance with one embodiment ofthe invention.

FIG. 13 is a flow chart diagram of a safe zone surgery performed using asurgical robot as described herein in accordance with one embodiment ofthe invention.

FIG. 14 is a flow chart diagram of a flexible catheter insertionprocedure performed using a surgical robot as described herein inaccordance with one embodiment of the invention.

FIG. 15A shows a screenshot of a monitor display showing a set up of theanatomy in X, Y and Z views in accordance with one embodiment of theinvention.

FIG. 15B shows a screenshot of a monitor display showing what the userviews during an invasive procedure in accordance with one embodiment ofthe invention.

FIG. 16 depicts a surgical robot having a plurality of optical markersmounted for tracking movement in an x-direction in accordance with oneembodiment of the invention.

FIGS. 17A-17B depict surgical instruments having a stop mechanism inaccordance with one embodiment of the invention.

FIGS. 17C-17E illustrate tools for manually adjusting a drill stop withreference to drill bit markings in accordance with one embodiment of theinvention.

FIGS. 17F-J illustrate tools for locking and holding a drill bit in aset position in accordance with one embodiment of the invention.

FIGS. 18A-18B depicts an end-effectuator having a clearance mechanism inaccordance with one embodiment of the invention.

FIG. 19A-19B depicts an end-effectuator having an attachment element forapplying distraction and/or compression forces in accordance with oneembodiment of the invention.

FIGS. 20A-20E show the use of calibration frames with the guidancesystem in accordance with one embodiment of the invention.

FIG. 21A depicts flexible roll configurations of a targeting fixture inaccordance with one embodiment of the invention.

FIG. 21B shows possible positions of markers along a line in space inaccordance with one embodiment of the invention.

FIG. 21C depicts flexible roll configurations of a targeting fixture inaccordance with one embodiment of the invention.

FIG. 21D shows a fixture that can be employed to provide desiredstiffness to the unrolled fixture such that it maintains its positionafter unrolling occurs in accordance with one embodiment of theinvention.

FIGS. 22A-22D depict a targeting fixture and method configured forapplication to the skull of a patient in accordance with one embodimentof the invention.

FIG. 23 depicts a dynamic tracking device mounted to the spinous processof the lumbar spine of a patient in accordance with one embodiment ofthe invention.

FIGS. 24-33 illustrate methods in accordance with one embodiment of theinvention.

FIG. 34 illustrates a computing platform that enables implementation ofvarious embodiments of the invention.

FIGS. 35A-35B display a surgical robot in accordance with one embodimentof the invention.

FIG. 36 illustrates a surgical robot system having a surveillance markerin accordance with one or more embodiments described herein.

FIG. 37 illustrates an example of a methodology for tracking a visualpoint on a rigid body using an array of three attached markers inaccordance with one embodiment of the invention.

FIG. 38 illustrates a procedure for monitoring the location of a pointof interest relative to three markers based on images received form themethodology illustrated in FIG. 37.

FIGS. 39A-F illustrates examples of tracking methodology based on anarray of three attached markers in accordance with one embodiment of theinvention.

FIG. 40 illustrates an example of a two dimensional representation forrotation about the Y-axis in accordance with one embodiment of theinvention.

FIG. 41A illustrates an alternative representation of a two dimensionalrepresentation for rotation about an X-axis in accordance with oneembodiment of the invention.

FIG. 41B illustrates an alternative representation of a two dimensionalrepresentation for rotation about a Y-axis in accordance with oneembodiment of the invention.

FIG. 41C illustrates an alternative representation of a two dimensionalrepresentation for rotation about a Z-axis in accordance with oneembodiment of the invention.

FIG. 42 provides a depiction of a noise within a frame of data.

FIG. 43 illustrates the depiction of a noise within a frame of data asshown in FIG. 42 with a stored point of interest.

FIG. 44 illustrates a depiction of results of applying a least squaresfitting algorithm for establishing a reference frame and transformingmarkers in accordance with one embodiment of the invention.

FIG. 45 illustrates a depiction of results of applying a least squaresfitting algorithm for establishing a reference frame and transformingmarkers as shown in FIG. 44 including noise.

FIG. 46 illustrates a depiction of error calculation for reference framemarkers in accordance with one embodiment of the invention.

FIG. 47 illustrates a graphical representation of methods of trackingthree dimensional movement of a rigid body.

FIG. 48 shows a perspective view illustrating a bayonet mount used toremovably couple the surgical instrument to the end-effectuator inaccordance with one embodiment of the invention.

FIGS. 49A-49F depict illustrations of targeting fixtures in accordancewith one embodiment of the invention.

FIG. 50A shows an example illustration of one portion of a spine withmarkers in accordance with one embodiment of the invention.

FIGS. 50B-50D show various illustrations of one portion of a spine withtwo independent trackers with markers in accordance with one embodimentof the invention.

FIG. 50E illustrates a representation of a display of a portion of aspine based on the location of a tracker in accordance with oneembodiment of the invention.

FIG. 50F illustrates a representation of a display of a portion of aspine based on the location of a tracker in accordance with oneembodiment of the invention.

FIGS. 50G-50H represent images of segmented CT scans in accordance withone embodiment of the invention.

FIG. 51 shows an example of a fixture for use with fluoroscopic views inaccordance with one embodiment of the invention.

FIGS. 52A-52B illustrates expected images on anteroposterior and lateralx-rays of the spine with a misaligned fluoroscopy (x-ray) machine inaccordance with one embodiment of the invention.

FIGS. 53A-53B illustrates expected images on anteroposterior and lateralx-rays of the spine with a well aligned fluoroscopy (x-ray) machine inaccordance with one embodiment of the invention.

FIGS. 54A-54B illustrates expected images on anteroposterior and lateralx-rays of the spine with a well aligned fluoroscopy (x-ray) machineincluding overlaid computer-generated graphical images showing theplanned trajectory and the current actual position of the robotend-effectuator in accordance with one embodiment of the invention.

FIGS. 55A-55B illustrates expected images on anteroposterior and lateralx-rays of the spine with a well aligned fluoroscopy (x-ray) machineshowing a feature on the targeting fixture designed to eliminateambiguity about directionality in accordance with one embodiment of theinvention.

FIG. 56 illustrates an axial view of a spine showing how a cartoonishaxial approximation of the spine can be constructed based on lateral andanteroposterior x-rays in accordance with one embodiment of theinvention.

FIGS. 57A-57B illustrates examples of targeting fixtures that facilitatedesired alignment of the targeting fixture relative to the x-ray imageplane in accordance with one embodiment of the invention.

FIGS. 58A-58B illustrates expected images on anteroposterior and lateralx-rays of the spine with a well aligned fluoroscopy (x-ray) machine whenparallax is present in accordance with one embodiment of the invention.

FIG. 59A illustrates two parallel plates with identically positionedradio-opaque markers in accordance with one embodiment of the invention.

FIG. 59B illustrates resulting expected x-ray demonstrating how markeroverlay is affected due to parallax using the two parallel plates asshown in FIG. 59A in accordance with one embodiment of the invention.

FIG. 60 shows a representation of the rendering of a computer screenwith an x-ray image that is affected by parallax overlaid by graphicalmarkers over the radio-opaque markers on two plates that have the samegeometry in accordance with one embodiment of the invention.

FIG. 61 shows a graphical overlay for the x-ray image screen intended tohelp the user physically line up the x-ray machine to get a view inwhich the markers on the two calibration plates shown in FIG. 59A in thecase where parallax complicates the view in accordance with oneembodiment of the invention.

FIG. 62 illustrates a method in accordance with at least one embodimentof the invention.

FIGS. 63A-63C illustrates various embodiments of an end-effectuatorincluding a modified mount with a clamping piece in accordance with atleast one embodiment of the invention.

FIGS. 64-65 illustrate embodiments of clamping piece actuation on aspinous process in accordance with some embodiments of the invention.

FIG. 66A illustrates a clamping piece modified with a targeting fixtureincluding a temporary marker skirt in accordance with at least oneembodiment of the invention.

FIG. 66B illustrates a clamping piece modified with a targeting fixtureas shown in FIG. 66A with the temporary marker skirt detached inaccordance with at least one embodiment of the invention.

FIG. 67 shows a modified Mayfield frame 6700 including one possibleconfiguration for active and radio-opaque markers in accordance with oneembodiment of the invention.

FIG. 68 shows end-effectuator 30 that includes nested dilators inaccordance with at least one embodiment of the invention.

FIGS. 69A-69C illustrates various embodiments of an end-effectuatorincluding cylindrical dilator tubes in accordance with at least oneembodiment of the invention.

FIG. 70 illustrates a method in accordance with at least one embodimentof the invention.

FIG. 71A illustrates a robot end-effectuator coupled with a curved guidetube for use with a curved or straight wire or tool in accordance withat least one embodiment of the invention.

FIG. 71B illustrates a robot end-effectuator coupled with a straightguide tube for use with a curved or straight wire or tool in accordancewith at least one embodiment of the invention.

FIG. 72 illustrates a guide tube in accordance with at least oneembodiment of the invention.

FIG. 73 illustrates a steerable and trackable needle in accordance withat least one embodiment of the invention.

FIG. 74 illustrates one embodiment of intersecting and interlocking bonescrews in accordance with at least one embodiment of the invention.

FIG. 75A-75B illustrates configurations of a robot for positioningalongside a bed of a patient that includes a targeting fixture coupledto an end-effectuator using a snap-in post.

FIG. 76 illustrates a surgical robot having a plurality of opticalmarkers mounted for calibration and tracking movement in accordance withone embodiment of the invention.

FIG. 77 illustrates a CT scan and methods in accordance with oneembodiment of the invention.

FIG. 78 illustrates a biopsy tool in accordance with one embodiment ofthe invention.

FIG. 79 illustrates a deep brain stimulation electrode placement methodperformed by the robot system in accordance with one embodiment of theinvention.

FIG. 80 illustrates a partial view of a surgical robot system includinga visual indicator comprising lights projected on the surgical field inaccordance with one embodiment of the invention.

FIG. 81 illustrates a perspective view of a robot system including acamera arm in accordance with one embodiment of the invention.

FIG. 82A illustrates a front-side perspective view of a robot systemincluding a camera arm in a stored position in accordance with oneembodiment of the invention.

FIG. 82B illustrates a rear-side perspective view of a robot systemincluding a camera arm in a stored position in accordance with oneembodiment of the invention.

FIG. 83 shows a lateral illustration of a patient lying supine, showingthe normal relative positions of the prostate, rectum, bladder, andpubic bone.

FIG. 84A shows a lateral illustration of a patient lying supine, showinghow inflation of a balloon can cause anterior displacement of theprostate toward the pubic bone, and a controllable amount of compressionagainst the pubic bone in accordance with one embodiment of theinvention.

FIG. 84B shows a lateral illustration of a patient lying supine, showinghow shifting of a paddle in the rectum can cause anterior displacementof the prostate toward the pubic bone, and a controllable amount ofcompression against the pubic bone in accordance with one embodiment ofthe invention.

FIG. 85 shows a sketch of a targeting fixture and immobilization deviceto be used for tracking the prostate during image-guided surgicalprocedures in accordance with one embodiment of the invention.

FIG. 86 shows an illustration of the device as illustrated in FIG. 85,in place in the rectum with prostate compressed and immobilized andtracking markers visible protruding caudal to the rectum in accordancewith one embodiment of the invention.

FIG. 87 illustrates a demonstration of a fibre Bragg grating (“FBG”)interrogation technology with a flexible fiber optic cable in accordancewith one embodiment of the invention.

FIG. 88 illustrates a tracker attached to the surface of the skin of apatient and rigidly interconnected to a fiber optic probe to allowaccurate tracking of the prostate in accordance with one embodiment ofthe invention.

FIG. 89 illustrates the fiber optic probe as depicted in FIG. 88 withoptically visible and MRI visible markings in accordance with oneembodiment of the invention.

FIGS. 90-93 illustrate various embodiments of a fiber optic probetracking system to allow accurate tracking of the prostate forimage-guided therapy in accordance with one embodiment of the invention.

FIG. 94 illustrates one embodiment of a nerve sensing probe.

DETAILED DESCRIPTION

Before any embodiments of the invention are explained in detail, it isto be understood that the invention is not limited in its application tothe details of construction and the arrangement of components set forthin the following description or illustrated in the following drawings.The invention is capable of other embodiments and of being practiced orof being carried out in various ways. Also, it is to be understood thatthe phraseology and terminology used herein is for the purpose ofdescription and should not be regarded as limiting. The use of“including,” “comprising,” or “having” and variations thereof herein ismeant to encompass the items listed thereafter and equivalents thereofas well as additional items. Unless specified or limited otherwise, theterms “mounted,” “connected,” “supported,” and “coupled” and variationsthereof are used broadly and encompass both direct and indirectmountings, connections, supports, and couplings. Further, “connected”and “coupled” are not restricted to physical or mechanical connectionsor couplings.

The following discussion is presented to enable a person skilled in theart to make and use embodiments of the invention. Various modificationsto the illustrated embodiments will be readily apparent to those skilledin the art, and the generic principles herein can be applied to otherembodiments and applications without departing from embodiments of theinvention. Thus, embodiments of the invention are not intended to belimited to embodiments shown, but are to be accorded the widest scopeconsistent with the principles and features disclosed herein. Thefollowing detailed description is to be read with reference to thefigures, in which like elements in different figures have like referencenumerals. The figures, which are not necessarily to scale, depictselected embodiments and are not intended to limit the scope ofembodiments of the invention. Skilled artisans will recognize theexamples provided herein have many useful alternatives and fall withinthe scope of embodiments of the invention.

The present invention can be understood more readily by reference to thefollowing detailed description, examples, drawings, and claims, andtheir previous and following description. However, before the presentdevices, systems, and/or methods are disclosed and described, it is tobe understood that this invention is not limited to the specificdevices, systems, and/or methods disclosed unless otherwise specified,and, as such, can, of course, vary. It is also to be understood that theterminology used herein is for the purpose of describing particularaspects only and is not intended to be limiting.

The following description is provided as an enabling teaching of theinvention in its best, currently known embodiment. To this end, thoseskilled in the relevant art will recognize and appreciate that manychanges can be made to the various aspects of the invention describedherein, while still obtaining the beneficial results of the presentinvention. It will also be apparent that some of the desired benefits ofthe present invention can be obtained by selecting some of the featuresof the present invention without utilizing other features. Accordingly,those who work in the art will recognize that many modifications andadaptations to the present invention are possible and can even bedesirable in certain circumstances and are a part of the presentinvention. Thus, the following description is provided as illustrativeof the principles of the present invention and not in limitationthereof.

As used throughout, the singular forms “a,” “an” and “the” includeplural referents unless the context clearly dictates otherwise. Thus,for example, reference to “a delivery conduit” can include two or moresuch delivery conduits unless the context indicates otherwise.

As used herein, the terms “optional” or “optionally” mean that thesubsequently described event or circumstance may or may not occur, andthat the description includes instances where said event or circumstanceoccurs and instances where it does not.

In some embodiments, the disclosed devices and systems can compriseelements of the devices and systems described in U.S. Patent PublicationNos. 2007/0238985, 2008/0154389, and 2008/0215181, the disclosures ofwhich are incorporated herein by reference in their entireties.

As employed in this specification and annexed drawings, the terms“unit,” “component,” “interface,” “system,” “platform,” and the like areintended to include a computer-related entity or an entity related to anoperational apparatus with one or more specific functionalities, whereinthe computer-related entity or the entity related to the operationalapparatus can be either hardware, a combination of hardware andsoftware, software, or software in execution. One or more of suchentities are also referred to as “functional elements.” As an example, aunit may be, but is not limited to being, a process running on aprocessor, a processor, an object, an executable computer program, athread of execution, a program, a memory (e.g., a hard disc drive),and/or a computer. As another example, a unit can be an apparatus withspecific functionality provided by mechanical parts operated by electricor electronic circuitry which is operated by a software application or afirmware application executed by a processor, wherein the processor canbe internal or external to the apparatus and executes at least a part ofthe software or firmware application. In addition or in the alternative,a unit can provide specific functionality based on physical structure orspecific arrangement of hardware elements. As yet another example, aunit can be an apparatus that provides specific functionality throughelectronic functional elements without mechanical parts, the electronicfunctional elements can include a processor therein to execute softwareor firmware that provides at least in part the functionality of theelectronic functional elements. An illustration of such apparatus can becontrol circuitry, such as a programmable logic controller. Theforegoing example and related illustrations are but a few examples andare not intended to be limiting. Moreover, while such illustrations arepresented for a unit, the foregoing examples also apply to a component,a system, a platform, and the like. It is noted that in certainembodiments, or in connection with certain aspects or features thereof,the terms “unit,” “component,” “system,” “interface,” “platform” can beutilized interchangeably.

Throughout the description and claims of this specification, the word“comprise” and variations of the word, such as “comprising” and“comprises,” means “including but not limited to,” and is not intendedto exclude, for example, other additives, components, integers or steps.

Referring now to FIGS. 1 and 35A, some embodiments include a surgicalrobot system 1 is disclosed in a room 10 where a medical procedure isoccurring. In some embodiments, the surgical robot system 1 can comprisea surgical robot 15 and one or more positioning sensors 12. In thisaspect, the surgical robot 15 can comprise a display means 29 (includingfor example a display 150 shown in FIG. 10), and a housing 27. In someembodiments a display 150 can be attached to the surgical robot 15,whereas in other embodiments, a display means 29 can be detached fromsurgical robot 15, either within surgical room 10 or in a remotelocation. In some embodiments, the housing 27 can comprise a robot arm23, and an end-effectuator 30 coupled to the robot arm 23 controlled byat least one motor 160. For example, in some embodiments, the surgicalrobot system 1 can include a motor assembly 155 comprising at least onemotor (represented as 160 in FIG. 10). In some embodiments, theend-effectuator 30 can comprise a surgical instrument 35. In otherembodiments, the end-effectuator 30 can be coupled to the surgicalinstrument 35. As used herein, the term “end-effectuator” is usedinterchangeably with the terms “end-effectuator,” “effectuator element,”and “effectuator element.” In some embodiments, the end-effectuator 30can comprise any known structure for effecting the movement of thesurgical instrument 35 in a desired manner.

In some embodiments, prior to performance of an invasive procedure, athree-dimensional (“3D”) image scan can be taken of a desired surgicalarea of the patient 18 and sent to a computer platform in communicationwith surgical robot 15 as described herein (see for example the platform3400 including the computing device 3401 shown in FIG. 34). In someembodiments, a physician can then program a desired point of insertionand trajectory for surgical instrument 35 to reach a desired anatomicaltarget within or upon the body of patient 18. In some embodiments, thedesired point of insertion and trajectory can be planned on the 3D imagescan, which in some embodiments, can be displayed on display means 29.In some embodiments, a physician can plan the trajectory and desiredinsertion point (if any) on a computed tomography scan (hereinafterreferred to as “CT scan”) of a patient 18. In some embodiments, the CTscan can be an isocentric C-arm type scan, an O-arm type scan, orintraoperative CT scan as is known in the art. However, in someembodiments, any known 3D image scan can be used in accordance with theembodiments of the invention described herein.

In some embodiments, the surgical robot system 1 can comprise a localpositioning system (“LPS”) subassembly to track the position of surgicalinstrument 35. The LPS subassembly can comprise at least oneradio-frequency (RF) transmitter 120 that is coupled were affixed to theend-effectuator 30 or the surgical instrument 35 at a desired location.In some embodiments, the at least one RF transmitter 120 can comprise aplurality of transmitters 120, such as, for example, at least three RFtransmitters 120. In another embodiment, the LPS subassembly cancomprise at least one RF receiver 110 configured to receive one or moreRF signals produced by the at least one RF transmitter 120. In someembodiments, the at least one RF receiver 110 can comprise a pluralityof RF receivers 110, such as, for example, at least three RF receivers110. In these embodiments, the RF receivers 110 can be positioned atknown locations within the room 10 where the medical procedure is totake place. In some embodiments, the RF receivers 110 can be positionedat known locations within the room 10 such that the RF receivers 110 arenot coplanar within a plane that is parallel to the floor of the room10.

In some embodiments, during use, the time of flight of an RF signal fromeach RF transmitter 120 of the at least one RF transmitter 120 to eachRF receiver 110 of the at least one RF receiver 110 (e.g., one RFreceiver, two RF receivers, three RF receivers, etc.) can be measured tocalculate the position of each RF transmitter 120. Because the velocityof the RF signal is known, the time of flight measurements result in atleast three distance measurements for each RF transmitter 120 (one toeach RF receiver 110).

In some embodiments, the surgical robot system 1 can comprise a controldevice (for example a computer 100 having a processor and a memorycoupled to the processor). In some embodiments, the processor of thecontrol device 100 can be configured to perform time of flightcalculations as described herein. Further, in some embodiments, can beconfigured to provide a geometrical description of the location of theat least one RF transmitter 120 with respect to an operative end of thesurgical instrument 35 or end-effectuator 30 that is utilized to performor assist in performing an invasive procedure. In some furtherembodiments, the position of the RF transmitter 120, as well as thedimensional profile of the surgical instrument 35 or the effectuatorelement 30 can be displayed on a monitor (for example on a display means29 such as the display 150 shown in FIG. 10). In one embodiment, theend-effectuator 30 can be a tubular element (for example a guide tube50) that is positioned at a desired location with respect to, forexample, a patient's 18 spine to facilitate the performance of a spinalsurgery. In some embodiments, the guide tube 50 can be aligned with thez axis 70 defined by a corresponding robot motor 160 or, for example,can be disposed at a selected angle relative to the z-axis 70. In eithercase, the processor of the control device (i.e. the computer 100) can beconfigured to account for the orientation of the tubular element and theposition of the RF transmitter 120. As further described herein, in someembodiments, the memory of the control device (computer 100 for example)can store software for performing the calculations and/or analysesrequired to perform many of the surgical method steps set forth herein.

Another embodiment of the disclosed surgical robot system 1 involves theutilization of a robot 15 that is capable of moving the end-effectuator30 along x-, y-, and z-axes (see 66, 68, 70 in FIG. 35B). In thisembodiment, the x-axis 66 can be orthogonal to the y-axis 68 and z-axis70, the y-axis 68 can be orthogonal to the x-axis 66 and z-axis 70, andthe z-axis 70 can be orthogonal to the x-axis 66 and the y-axis 68. Insome embodiments, the robot 15 can be configured to effect movement ofthe end-effectuator 30 along one axis independently of the other axes.For example, in some embodiments, the robot 15 can cause theend-effectuator 30 to move a given distance along the x-axis 66 withoutcausing any significant movement of the end-effectuator 30 along they-axis 68 or z-axis 70.

In some further embodiments, the end-effectuator 30 can be configuredfor selective rotation about one or more of the x-axis 66, y-axis 68,and z-axis 70 (such that one or more of the Cardanic Euler Angles (e.g.,roll, pitch, and/or yaw) associated with the end-effectuator 30 can beselectively controlled). In some embodiments, during operation, theend-effectuator 30 and/or surgical instrument 35 can be aligned with aselected orientation axis (labeled “Z Tube” in FIG. 35B) that can beselectively varied and monitored by an agent (for example computer 100and platform 3400) that can operate the surgical robot system 1. In someembodiments, selective control of the axial rotation and orientation ofthe end-effectuator 30 can permit performance of medical procedures withsignificantly improved accuracy compared to conventional robots thatutilize, for example, a six degree of freedom robot arm 23 comprisingonly rotational axes.

In some embodiments, as shown in FIG. 1, the robot arm 23 that can bepositioned above the body of the patient 18, with the end-effectuator 30selectively angled relative to the z-axis toward the body of the patient18. In this aspect, in some embodiments, the robotic surgical system 1can comprise systems for stabilizing the robotic arm 23, theend-effectuator 30, and/or the surgical instrument 35 at theirrespective positions in the event of power failure. In some embodiments,the robotic arm 23, end-effectuator 30, and/or surgical instrument 35can comprise a conventional worm-drive mechanism (not shown) coupled tothe robotic arm 23, configured to effect movement of the robotic armalong the z-axis 70. In some embodiments, the system for stabilizing therobotic arm 23, end-effectuator 30, and/or surgical instrument 35 cancomprise a counterbalance coupled to the robotic arm 23. In anotherembodiment, the means for maintaining the robotic arm 23,end-effectuator 30, and/or surgical instrument 35 can comprise aconventional brake mechanism (not shown) that is coupled to at least aportion of the robotic arm 23, such as, for example, the end-effectuator30, and that is configured for activation in response to a loss of poweror “power off” condition of the surgical robot 15.

Referring to FIG. 1, in some embodiments, the surgical robot system 1can comprise a plurality of positioning sensors 12 configured to receiveRF signals from the at least one conventional RF transmitter (not shown)located within room 10. In some embodiments, the at least one RFtransmitter 120 can be disposed on various points on the surgical robot15 and/or on patient 18. For example, in some embodiments, the at leastone RF transmitter 120 can be attached to one or more of the housing 27,robot arm 23, end-effectuator 30, and surgical instrument 35. Someembodiments include positioning sensors 12 that in some embodimentscomprise RF receivers 110. In some embodiments, RF receivers 110 are incommunication with a computer platform as described herein (see forexample 3400 comprising a computing device 3401 FIG. 34) that receivesthe signal from the RF transmitters 120. In some embodiments, eachtransmitter 120 of the at least one RF transmitter 120 can transmit RFenergy on a different frequency so that the identity of each transmitter120 in the room 10 can be determined. In some embodiments, the locationof the at least one RF transmitter 120, and, consequently, the objectsto which the transmitters 120 are attached, are calculated by thecomputer (e.g., computing device 3401 in FIG. 34) using time-of-flightprocesses.

In some embodiments, the computer (not shown in FIG. 1) is also incommunication with surgical robot 15. In some embodiments, aconventional processor (not shown) of the computer 100 of the computingdevice 3401 can be configured to effect movement of the surgical robot15 according to a preplanned trajectory selected prior to the procedure.For example, in some embodiments, the computer 100 of the computingdevice 3401 can use robotic guidance software 3406 and robotic guidancedata storage 3407 (shown in FIG. 34) to effect movement of the surgicalrobot 15.

In some embodiments, the position of surgical instrument 35 can bedynamically updated so that surgical robot 15 is aware of the locationof surgical instrument 35 at all times during the procedure.Consequently, in some embodiments, the surgical robot 15 can move thesurgical instrument 35 to the desired position quickly, with minimaldamage to patient 18, and without any further assistance from aphysician (unless the physician so desires). In some furtherembodiments, the surgical robot 15 can be configured to correct the pathof surgical instrument 35 if the surgical instrument 35 strays from theselected, preplanned trajectory.

In some embodiments, the surgical robot 15 can be configured to permitstoppage, modification, and/or manual control of the movement of theend-effectuator 30 and/or surgical instrument 35. Thus, in use, in someembodiments, an agent (e.g., a physician or other user) that can operatethe system 1 has the option to stop, modify, or manually control theautonomous movement of end-effectuator 30 and/or surgical instrument 35.Further, in some embodiments, tolerance controls can be preprogrammedinto the surgical robot 15 and/or processor of the computer platform3400 (such that the movement of the end-effectuator 30 and/or surgicalinstrument 35 is adjusted in response to specified conditions beingmet). For example, in some embodiments, if the surgical robot 15 cannotdetect the position of surgical instrument 35 because of a malfunctionin the at least one RF transmitter 120, then the surgical robot 15 canbe configured to stop movement of end-effectuator 30 and/or surgicalinstrument 35. In some embodiments, if surgical robot 15 detects aresistance, such as a force resistance or a torque resistance above atolerance level, then the surgical robot 15 can be configured to stopmovement of end-effectuator 30 and/or surgical instrument 35.

In some embodiments, the computer 100 for use in the system (for examplerepresented by computing device 3401), as further described herein, canbe located within surgical robot 15, or, alternatively, in anotherlocation within surgical room 10 or in a remote location. In someembodiments, the computer 100 can be positioned in operativecommunication with positioning sensors 12 and surgical robot 15.

In some further embodiments, the surgical robot 15 can also be used withexisting conventional guidance systems. Thus, alternative conventionalguidance systems beyond those specifically disclosed herein are withinthe scope and spirit of the invention. For instance, a conventionaloptical tracking system 3417 for tracking the location of the surgicaldevice, or a commercially available infrared optical tracking system3417, such as Optotrak® (Optotrak® is a registered trademark of NorthernDigital Inc. Northern Digital, Waterloo, Ontario, Canada), can be usedto track the patient 18 movement and the robot's base 25 location and/orintermediate axis location, and used with the surgical robot system 1.In some embodiments in which the surgical robot system 1 comprises aconventional infrared optical tracking system 3417, the surgical robotsystem 1 can comprise conventional optical markers attached to selectedlocations on the end-effectuator 30 and/or the surgical instrument 35that are configured to emit or reflect light. In some embodiments, thelight emitted from and/or reflected by the markers can be read bycameras (for example with cameras 8200 shown in FIG. 81) and/or opticalsensors and the location of the object can be calculated throughtriangulation methods (such as stereo-photogrammetry).

Referring now to FIG. 2, it is seen that, in some embodiments, thesurgical robot 15 can comprise a base 25 connected to wheels 31. Thesize and mobility of these embodiments can enable the surgical robot tobe readily moved from patient to patient and room to room as desired. Asshown, in some embodiments, the surgical robot 15 can further comprise acase 40 that is slidably attached to base 25 such that the case 40 canslide up and down along the z-axis 70 substantially perpendicular to thesurface on which base 25 sits. In some embodiments, the surgical robot15 can include a display means 29, and a housing 27 which contains robotarm 23.

As described earlier, the end-effectuator 30 can comprise a surgicalinstrument 35, whereas in other embodiments, the end-effectuator 30 canbe coupled to the surgical instrument 35. In some embodiments, it is arm23 can be connected to the end-effectuator 30, with surgical instrument35 being removably attached to the end-effectuator 30.

Referring now to FIGS. 2, 3A-3B, 4, 5A-5B, 6, 7, and 8A-8B, in someembodiments, the effectuator element 30 can include an outer surface 30d, and can comprise a distal end 30 a defining a beveled leading edge 30b and a non-beveled leading edge 30 c. In some embodiments, the surgicalinstrument 35 can be any known conventional instrument, device, hardwarecomponent, and/or attachment that is used during performance of a aninvasive or non-invasive medical procedure (including surgical,therapeutic, and diagnostic procedures). For example and withoutlimitation, in some embodiments, the surgical instrument 35 can beembodied in or can comprise a needle 7405, 7410, a conventional probe, aconventional screw, a conventional drill, a conventional tap, aconventional catheter, a conventional scalpel forceps, or the like. Inaddition or in the alternative, in some embodiments, the surgicalinstrument 35 can be a biological delivery device, such as, for exampleand without limitation, a conventional syringe, which can distributebiologically acting compounds throughout the body of a patient 18. Insome embodiments, the surgical instrument 35 can comprise a guide tube50 (also referred to herein as a “Z-tube 50”) that defines a centralbore configured for receipt of one or more additional surgicalinstruments 35.

In some embodiments, the surgical robot 15 is moveable in a plurality ofaxes (for instance x-axis 66, y-axis 68, and z-axis 70) in order toimprove the ability to accurately and precisely reach a target location.Some embodiments include a robot 15 that moves on a Cartesianpositioning system; that is, movements in different axes can occurrelatively independently of one another instead of at the end of aseries of joints.

Referring now to FIGS. 3A and 3B, the movement of case 40 relative tobase 25 of surgical robot 15 is represented as a change of height of thesystem 1 and the position of the case 40 with respect to the base 25. Asillustrated, in some embodiments, case 40 can be configured to be raisedand lowered relative to the base 25 along the z-axis. Some embodimentsinclude a housing 27 that can be attached to case 40 and be configuredto move in the z-direction (defined by z-frame 72) with case 40 whencase 40 is raised and lowered. Consequently, in some embodiments, arm23, the end-effectuator 30, and surgical instrument 35 can be configuredto move with case 40 as case 40 is raised and lowered relative to base25.

In a further embodiment, referring now to FIG. 4, housing 27 can beslidably attached to case 40 so that it can extend and retract along thex-axis 66 relative to case 40 and substantially perpendicularly to thedirection case 40 moves relative to base 25. Consequently, in someembodiments, the robot arm 23, the end-effectuator 30, and surgicalinstrument 35 can be configured to move with housing 27 as housing 27 isextended and retracted relative to case 40.

Referring now to FIGS. 5A and 5B, the extension of arm 23 along they-axis 68 is shown. In some embodiments, robot arm 23 can be extendablealong the y-axis 68 relative to case 40, base 25, and housing 27.Consequently, in some embodiments, the end-effectuator 30 and surgicalinstrument 35 can be configured to move with arm 23 as arm 23 isextended and retracted relative to housing 27. In some embodiments, arm23 can be attached to a low profile rail system (not shown) which isencased by housing 27.

Referring now to FIGS. 6, 7 and FIGS. 8A-B, the movement of theend-effectuator 30 is shown. FIG. 6 shows an embodiment of anend-effectuator 30 that is configured to rotate about the y-axis 68,performing a rotation having a specific roll 62. FIG. 7 shows anembodiment of an end-effectuator 30 that is configured to rotate aboutthe x-axis 66, performing a rotation having a specific pitch 60. FIG. 8shows an embodiment of an end-effectuator 30 that is configured to raiseand lower surgical instrument 35 along a substantially vertical axis,which can be a secondary movable axis 64, referred to as “Z-tube axis64”. In some embodiments, the orientation of the guide tube 50 can beinitially aligned with z-axis 70, but such orientation can change inresponse to changes in roll 62 and/or pitch 60.

FIG. 9 shows a system diagram of the 3D positioning sensors 110,computer 100, and RF transmitters 120 in accordance with someembodiments of the invention is provided. As shown, computer 100 is incommunication with positioning sensors 110. In some embodiments, duringoperation, RF transmitters 120 are attached to various points on thesurgical robot 15. In some embodiments, the RF transmitters 120 can alsobe attached to various points on or around an anatomical target of apatient 18. In some embodiments, computer 100 can be configured to senda signal to the RF transmitters 120, prompting the RF transmitters 120to transmit RF signals that are read by the positioning sensors 110. Insome embodiments, the computer 100 can be coupled to the RF transmitters120 using any conventional communication means, whether wired orwireless. In some embodiments, the positioning sensors 110 can be incommunication with computer 100, which can be configured to calculatethe location of the positions of all the RF transmitters 120 based ontime-of-flight information received from the positioning sensors 110. Insome embodiments, computer 100 can be configured to dynamically updatethe calculated location of the surgical instrument 35 and/orend-effectuator 30 being used in the procedure, which can be displayedto the agent.

Some embodiments can include a system diagram of surgical robot system 1having a computer 100, a display means 29 comprising a display 150, userinput 170, and motors 160, provided as illustrated in FIG. 10. In someembodiments, motors 160 can be installed in the surgical robot 15 andcontrol the movement of the end-effectuator 30 and/or surgicalinstrument 35 as described above. In some embodiments, computer 100 canbe configured to dynamically update the location of the surgicalinstrument 35 being used in the procedure, and can be configured to sendappropriate signals to the motors 160 such that the surgical robot 15has a corresponding response to the information received by computer100. For example, in some embodiments, in response to informationreceived by computer 100, the computer 100 can be configured to promptthe motors 160 to move the surgical instrument 35 along a preplannedtrajectory.

In some embodiments, prior to performance of a medical procedure, suchas, for example, an invasive surgical procedure, user input 170 can beused to plan the trajectory for a desired navigation. After the medicalprocedure has commenced, if changes in the trajectory and/or movement ofthe end-effectuator 30 and/or surgical instrument 35 are desired, a usercan use the user input 170 to input the desired changes, and thecomputer 100 can be configured to transmit corresponding signals to themotors 160 in response to the user input 170.

In some embodiments, the motors 160 can be or can comprise conventionalpulse motors. In this aspect, in some embodiments, the pulse motors canbe in a conventional direct drive configuration or a belt drive andpulley combination attached to the surgical instrument 35.Alternatively, in other embodiments, the motors 160 can be conventionalpulse motors that are attached to a conventional belt driverack-and-pinion system or equivalent conventional power transmissioncomponent.

In some embodiments, the use of conventional linear pulse motors withinthe surgical robot 15 can permit establishment of a non-rigid positionfor the end-effectuator 30 and/or surgical instrument 35. Thus, in someembodiments, the end-effectuator 30 and/or surgical instrument 35 willnot be fixed in a completely rigid position, but rather theend-effectuator 30 and/or the surgical instrument 35 can be configuredsuch that an agent (e.g., a surgeon or other user) can overcome thex-axis 66 and y-axis 68, and force the end-effectuator 30 and/orsurgical instrument 35 from its current position. For example, in someembodiments, the amount of force necessary to overcome such axes can beadjusted and configured automatically or by an agent. In someembodiments, the surgical robot 15 can comprise circuitry configured tomonitor one or more of: (a) the position of the robot arm 23, theend-effectuator 30, and/or the surgical instrument 35 along the x-axis66, y-axis 68, and z-axis 70; (b) the rotational position (e.g., roll 62and pitch 60) of the robot arm 23, the end-effectuator 30, and/or thesurgical instrument 35 relative to the x-(66), y-(68), and z-(70) axes;and (c) the position of the end-effectuator 30, and/or the surgicalinstrument 35 along the travel of the re-orientable axis that isparallel at all times to the end-effectuator 30 and surgical instrument35 (the Z-tube axis 64).

In one embodiment, circuitry for monitoring the positions of the x-axis66, y-axis 68, z-axis 70, Z-tube axis 64, roll 62, and/or pitch 60 cancomprise relative or absolute conventional encoder units (also referredto as encoders) embedded within or functionally coupled to conventionalactuators and/or bearings of at least one of the motors 160. Optionally,in some embodiments, the circuitry of the surgical robot 15 can beconfigured to provide auditory, visual, and/or tactile feedback to thesurgeon or other user when the desired amount of positional tolerance(e.g., rotational tolerance, translational tolerance, a combinationthereof, or the like) for the trajectory has been exceeded. In someembodiments, the positional tolerance can be configurable and defined,for example, in units of degrees and/or millimeters.

In some embodiments, the robot 15 moves into a selected position, readyfor the surgeon to deliver a selected surgical instrument 35, such as,for example and without limitation, a conventional screw, a biopsyneedle 8110, and the like. In some embodiments, as the surgeon works, ifthe surgeon inadvertently forces the end-effectuator 30 and/or surgicalinstrument 35 off of the desired trajectory, then the system 1 can beconfigured to provide an audible warning and/or a visual warning. Forexample, in some embodiments, the system 1 can produce audible beepsand/or display a warning message on the display means 29, such as“Warning: Off Trajectory,” while also displaying the axes for which anacceptable tolerance has been exceeded.

In some embodiments, in addition to, or in place of the audible warning,a light illumination may be directed to the end-effectuator 30, theguide tube 50, the operation area (i.e. the surgical field 17) of thepatient 18, or a combination of these regions. For example, someembodiments include at least one visual indication 900 capable ofilluminating a surgical field 17 of a patient 18. Some embodimentsinclude at least one visual indication 900 capable of indicating atarget lock by projecting an illumination on a surgical field 17. Insome embodiments, the system 1 can provide feedback to the userregarding whether the robot 15 is locked on target. In some otherembodiments, the system 1 can provide an alert to the user regardingwhether at least one marker 720 is blocked, or whether the system 1 isactively seeking one or more markers 720.

In some embodiments, the visual indication 900 can be projected by oneor more conventional light emitting diodes mounted on or near the robotend-effectuator 30. In some embodiments, the visual indication cancomprise lights projected on the surgical field 17 including a colorindicative of the current situation (see for example, FIG. 80). In someembodiments, a green projected light could represent a locked-on-targetsituation, whereas in some embodiments, a red illumination couldindicate a trajectory error, or obscured markers 720. In some otherembodiments, a yellow illumination could indicate the system 1 isactively seeking one or more markers 720.

In some embodiments, if the surgeon attempts to exceed the acceptabletolerances, the robot 15 can be configured to provide mechanicalresistance (“push back” or haptic feedback) to the movement of theend-effectuator 30 and/or surgical instrument 35 in this manner, therebypromoting movement of the end-effectuator 30 and/or surgical instrument35 back to the correct, selected orientation. In some embodiments, whenthe surgeon then begins to correct the improper position, the robot 15can be configured to substantially immediately return theend-effectuator 30 and/or surgical instrument 35 back to the desiredtrajectory, at which time the audible and visual warnings and alerts canbe configured to cease. For example, in some embodiments, the visualwarning could include a visual indication 900 that may include a greenlight if no tolerances have been exceeded, or a red light if tolerancesare about to, or have been exceeded.

As one will appreciate, a conventional worm-drive system would beabsolutely rigid, and a robot 15 having such a worm-drive system wouldbe unable to be passively moved (without breaking the robot 15) nomatter how hard the surgeon pushed. Furthermore, a completely rigidarticulation system can be inherently unsafe to a patient 18. Forexample, if such a robot 15 were moving toward the patient 18 andinadvertently collided with tissues, then these tissues could bedamaged. Although conventional sensors can be placed on the surface ofsuch a robot 15 to compensate for these risks, such sensors can addconsiderable complexity to the overall system 1 and would be difficultto operate in a fail-safe mode. In contrast, during use of the robot 15described herein, if the end-effectuator 30 and/or surgical instrument35 inadvertently collides with tissues of the patient 18, a collisionwould occur with a more tolerable force that would be unlikely to damagesuch tissues. Additionally, in some embodiments, auditory and/or visualfeedback as described above can be provided to indicate an increase inthe current required to overcome the obstacle. Furthermore, in someembodiments, the end-effectuator 30 of the robot 15 can be configured todisplace itself (move away) from the inadvertently contacted tissue if athreshold required motor 160 current is encountered. In someembodiments, this threshold could be configured (by a control component,for example) for each axis such that the moderate forces associated withengagement between the tissue and the end-effectuator 30 can berecognized and/or avoided.

In some embodiments, the amount of rigidity associated with thepositioning and orientation of the end-effectuator 30 and/or thesurgical instrument 35 can be selectively varied. For example, in someembodiments, the robot 15 can be configured to shift between ahigh-rigidity mode and a low-rigidity mode. In some embodiments, therobot 15 can be programmed so that it automatically shifts to thelow-rigidity mode as the end-effectuator 30 and surgical instrument 35are shifted from one trajectory to another, from a starting position asthey approach a target trajectory and/or target position. Moreover, insome embodiment, once the end-effectuator 30 and/or surgical instrument35 is within a selected distance of the target trajectory and/or targetposition, such as, for example, within about 1° and about 1 mm of thetarget, the robot 15 can be configured to shift to the high-rigiditymode. In some embodiments, this mechanism may improve safety because therobot 15 would be unlikely to cause injury if it inadvertently collidedwith the patient 18 while in the low-rigidity mode.

Some embodiments include a robot 15 that can be configured to effectmovement of the end-effectuator 30 and/or surgical instrument 35 in aselected sequence of distinct movements. In some embodiments, duringmovement of the end-effectuator 30 and/or surgical instrument 35 fromone trajectory to another trajectory, the x-axis 66, y-axis 68, roll 62,and 60 pitch 60 orientations are all changed simultaneously, and thespeed of movement of the end-effectuator 30 can be increased.Consequently, because of the range of positions through which theend-effectuator 30 travels, the likelihood of a collision with thetissue of the patient 18 can also be increased. Hence, in someembodiments, the robot 15 can be configured to effect movement of theend-effectuator 30 and/or surgical instrument 35 such that the positionof the end-effectuator 30 and/or surgical instrument 35 within thex-axis 66 and the y-axis 68 are adjusted before the roll 62 and pitch 60of the end-effectuator 30 and/or surgical instrument 35 are adjusted. Insome alternative embodiments, the robot 15 can be configured to effectmovement of the end-effectuator 30 and/or surgical instrument 35 so thatthe roll 62 and pitch 60 are shifted to 0°. The position of theend-effectuator 30 and/or surgical instrument 35 within the x-axis 66and the y-axis 68 are adjusted, and then the roll 62 and pitch 60 of theend-effectuator 30 and/or surgical instrument 35 are adjusted.

Some embodiments include a robot 15 that can be optionally configured toensure that the end-effectuator 30 and/or surgical instrument 35 aremoved vertically along the z-axis 70 (away from the patient 18) by aselected amount before a change in the position and/or trajectory of theend-effectuator 30 and/or surgical instrument 35 is effected. Forexample, in some embodiments, when an agent (for example, a surgeon orother user, or equipment) changes the trajectory of the end-effectuator30 and/or surgical instrument 35 from a first trajectory to a secondtrajectory, the robot 15 can be configured to vertically displace theend-effectuator 30 and/or surgical instrument 35 from the body of thepatient 18 along the z-axis 70 by the selected amount (while adjustingx-axis 66 and y-axis 68 configurations to remain on the first trajectoryvector, for example), and then effecting the change in position and/ororientation of the end-effectuator 30 and/or surgical instrument 35.This ensures that the end-effectuator 30 and/or surgical instrument 35do not move laterally while embedded within the tissue of the patient18. Optionally, in some embodiments, the robot 15 can be configured toproduce a warning message that seeks confirmation from the agent (forexample, a surgeon or other user, or equipment) that it is safe toproceed with a change in the trajectory of the end-effectuator 30 and/orsurgical instrument 35 without first displacing the end-effectuator 30and/or surgical instrument 35 along the z-axis.

In some embodiments, at least one conventional force sensor (not shown)can be coupled to the end-effectuator 30 and/or surgical instrument 35such that the at least one force sensor receives forces applied alongthe orientation axis (Z-tube axis 64) to the surgical instrument 35. Insome embodiments, the at least one force sensor can be configured toproduce a digital signal. In some embodiments for example, the digitalsignal can be indicative of the force that is applied in the directionof the Z-tube axis 64 to the surgical instrument 35 by the body of thepatient 18 as the surgical instrument 35 advances into the tissue of thepatient 18. In some embodiments, the at least one force sensor can be asmall conventional uniaxial load cell based on a conventional straingauge mechanism. In some embodiments, the uniaxial load cell can becoupled to, for example, analog-to-digital filtering to supply acontinuous digital data stream to the system 1. Optionally, in someembodiments, the at least one force sensor can be configured tosubstantially continuously produce signals indicative of the force thatis currently being applied to the surgical instrument 35. In someembodiments, the surgical instrument 35 can be advanced into the tissueof the patient 18 by lowering the z-axis 70 while the position of theend-effectuator 30 and/or surgical instrument 35 along the x-axis 66 andy-axes 68 is adjusted such that alignment with the selected trajectoryvector is substantially maintained. Furthermore, in some embodiments,the roll 62 and pitch 60 orientations can remain constant or self-adjustduring movement of the x-(66), y-(68), and z-(70) axes such that thesurgical instrument 35 remains oriented along the selected trajectoryvector. In some embodiments, the position of the end-effectuator 30along the z-axis 70 can be locked at a selected mid-range position(spaced a selected distance from the patient 18) as the surgicalinstrument 35 advances into the tissue of the patient 18. In someembodiments, the stiffness of the end-effectuator 30 and/or the surgicalinstrument 35 can be set at a selected level as further describedherein. For example, in some embodiments, the stiffness of the Z-tubeaxis 64 position of the end-effectuator 30 and/or the surgicalinstrument 35 can be coupled to a conventional mechanical lock (notshown) configured to impart desired longitudinal stiffnesscharacteristics to the end-effectuator 30 and/or surgical instrument 35.In some embodiments, if the end-effectuator 30 and/or surgicalinstrument 35 lack sufficient longitudinal stiffness, then thecounterforce applied by the tissue of the patient 18 during penetrationof the surgical instrument 35 can oppose the direction of advancement ofthe surgical instrument 35 such that the surgical instrument 35 cannotadvance along the selected trajectory vector. In other words, as thez-axis 70 advances downwards, the Z-tube axis 64 can be forced up andthere can be no net advancement of the surgical instrument 35. In someembodiments, the at least one force sensor can permit an agent (forexample, a surgeon or other user, or equipment) to determine, (based onsudden increase in the level of applied force monitored by the forcesensor at the end-effectuator 30 and/or the surgical instrument 35),when the surgical instrument 35 has encountered a bone or other specificstructure within the body of the patient 18.

In some alternative embodiments, the orientation angle of theend-effectuator 30 and/or surgical instrument 35 and the x-axis 66 andy-axis 68 can be configured to align the Z-tube axis 64 with the desiredtrajectory vector at a fully retracted Z-tube position, while a z-axis70 position is set in which the distal tip of the surgical instrument 35is poised to enter tissue. In this configuration, in some embodiments,the end-effectuator 30 can be positioned in a manner that theend-effectuator 30 can move, for example, exactly or substantiallyexactly down the trajectory vector if it were advanced only along guidetube 50. In such scenario, in some embodiments, advancing the Z-tubeaxis 64 can cause the guide tube 50 to enter into tissue, and an agent(a surgeon or other user, equipment, etc.) can monitor change in forcefrom the load sensor. Advancement can continue until a sudden increasein applied force is detected at the time the surgical instrument 35contacts bone.

In some embodiments, the robot 15 can be configured to deactivate theone or more motors 160 that advance the Z-tube axis 64 such that theend-effectuator 30 and/or the surgical instrument 35 can move freely inthe Z-tube axis 64 direction while the position of the end-effectuator30 and/or the surgical instrument 35 continues to be monitored. In someembodiments, the surgeon can then push the end-effectuator 30 down alongthe Z-tube axis 64, (which coincides with the desired trajectory vector)by hand. In some embodiments, if the end-effectuator 30 position hasbeen forced out of alignment with the trajectory vector, the position ofthe surgical instrument 35 can be corrected by adjustment along thex-(66) and/or y-(68) axes and/or in the roll 62 and/or pitch 60directions. In some embodiments, when motor 160 associated with theZ-tube 50 movement of the surgical instrument 35 is deactivated, theagent (for example, a surgeon or other user, or equipment) can manuallyforce the surgical instrument 35 to advance until a tactile sense of thesurgical instrument 35 contacts bone, or another known region of thebody).

In some further embodiments, the robotic surgical system 1 can comprisea plurality of conventional tracking markers 720 configured to track themovement of the robot arm 23, the end-effectuator 30, and/or thesurgical instrument 35 in three dimensions. It should be appreciatedthat three dimensional positional information from tracking markers 720can be used in conjunction with the one dimensional linear positionalinformation from absolute or relative conventional linear encoders oneach axis of the robot 15 to maintain a high degree of accuracy. In someembodiments, the plurality of tracking markers 720 can be mounted (orotherwise secured) thereon an outer surface of the robot 15, such as,for example and without limitation, on the base 25 of the robot 15, orthe robot arm 23. In some embodiments, the plurality of tracking markers720 can be configured to track the movement of the robot 15 arm, theend-effectuator 30, and/or the surgical instrument 35. In someembodiments, the computer 100 can utilize the tracking information tocalculate the orientation and coordinates of the distal tip 30 a of thesurgical instrument 35 based on encoder counts along the x-axis 66,y-axis 68, z-axis 70, the Z-tube axis 64, and the roll 62 and pitch 60axes. Further, in some embodiments, the plurality of tracking markers720 can be positioned on the base 25 of the robot 15 spaced from thesurgical field 17 to reduce the likelihood of being obscured by thesurgeon, surgical tools, or other parts of the robot 15. In someembodiments, at least one tracking marker 720 of the plurality oftracking markers 720 can be mounted or otherwise secured to theend-effectuator 30. In some embodiments, the positioning of one or moretracking markers 720 on the end-effectuator 30 can maximize the accuracyof the positional measurements by serving to check or verify theend-effectuator 30 position (calculated from the positional informationfrom the markers on the base 25 of the robot 15 and the encoder countsof the x-(66), y-(68), roll 62, pitch 60, and Z-tube axes 64).

In some further embodiments, at least one optical marker of theplurality of optical tracking markers 720 can be positioned on the robot15 between the base 25 of the robot 15 and the end-effectuator 30instead of, or in addition to, the markers 720 on the base 25 of therobot 15, (see FIG. 16). In some embodiments, the at least one trackingmarker 720 can be mounted to a portion of the robot 15 that effectsmovement of the end-effectuator 30 and/or surgical instrument 35 alongthe x-axis to enable the tracking marker 720 to move along the x-axis 66as the end-effectuator 30 and surgical instrument 35 move along thex-axis 66 (see FIG. 76). The placement of the tracking markers 720 inthis way can reduce the likelihood of a surgeon blocking the trackingmarker 720 from the cameras or detection device, or the tracking marker720 becoming an obstruction to surgery. In certain embodiments, becauseof the high accuracy in calculating the orientation and position of theend-effectuator 30 based on the tracking marker 720 outputs and/orencoder counts from each axis, it can be possible to very accuratelydetermine the position of the end-effectuator 30. For example, in someembodiments, without requiring knowledge of the counts of axis encodersfor the z-axis 70, which is between the x-axis 66 and the base 25,knowing only the position of the markers 720 on the x-axis 66 and thecounts of encoders on the y-(68), roll 62, pitch 60, and Z-tube axes 64can enable computation of the position of the end-effectuator 30. Insome embodiments, the placement of markers 720 on any intermediate axisof the robot 15 can permit the exact position of the end-effectuator 30to be calculated based on location of such markers 720 and counts ofencoders on axes (66, 62, 60, 64) between the markers 720 and theend-effectuator 30. In some embodiments, from the configuration of therobot 15 (see for example, FIG. 2), the order of axes from the base 25to the end-effectuator 30 is z-(70) then x-(66) then y-(68) then roll 62then pitch 60 then Z-tube 64. Therefore, for example, within embodimentsin which tracking markers 720 are placed on the housing 27 of the robot15 that moves with the roll 62 axis, the locations of such trackingmarkers 720 and the encoder counts of the pitch 60 and Z-tube axes 64can be sufficient to calculate the end-effectuator 30 position.

In some embodiments, when the surgical instrument 35 is advanced intothe tissue of the patient 18 with the assistance of a guide tube 50, thesurgical instrument 35 can comprise a stop mechanism 52 that isconfigured to prevent the surgical instrument 35 from advancing when itreaches a predetermined amount of protrusion (see for example, FIGS.17A-B). In some embodiments, by knowing the lengths of the guide tube 50and the surgical instrument 35, the distance between the respective endsof the surgical instrument 35, and the location where the stop mechanism52 is attached, it is possible to determine the maximum distance pastthe end of the guide tube 50 that the surgical instrument 35 canprotrude.

In some embodiments, it can be desirable to monitor not just the maximumprotrusion distance of the surgical instrument 35, but also the actualprotrusion distance at any instant during the insertion process.Therefore, in some embodiments, the robot 15 can substantiallycontinuously monitor the protrusion distance, and in some embodiments,the distance can be displayed on a display (such as display means 29).In some embodiments, protrusion distance can be substantiallycontinuously monitored using a spring-loaded plunger 54 including aspring-loaded mechanism 55 a and sensor pad 55 b that has a coupledwiper 56 (see for example FIG. 17B). In some embodiments, the stopmechanism 52 on the surgical instrument 35 can be configured to contactthe spring-loaded mechanism 55 well before it encounters the end of theguide tube 50. In some embodiments, when the wiper 56 moves across theposition sensor pad 55 b, its linear position is sampled, therebypermitting calculation of the distance by which the surgical instrument35 protrudes past the end of the guide tube 50 substantially inreal-time. In some embodiments, any conventional linear encodingmechanism can be used to monitor the plunger's depth of depression andtransmit that information to the computer 100 as further describedherein.

Some embodiments include instruments that enable the stop on a drill bit42 to be manually adjusted with reference to markings 44 on the drillbit 42. For example, FIGS. 17C-17E depict tools for manually adjusting adrill stop 46 with reference to drill bit markings 44 in accordance withone embodiment of the invention. As shown, in some embodiments, thedrill bit 42 can include release mechanisms 48 on each end of the drillstop 46. In some embodiments, if the release 48 on one end of the drillstop 46 is pulled, it is possible to move the drill stop 46 up the shaftof the drill bit 42. In some embodiments, if the release 48 on the otherend of the drill stop 46 is pulled, it is possible to move the drillstop 46 down the shaft (see the direction of movement in FIGS. 17D and17E). In some embodiments, if neither release mechanism 48 is pulled,the drill stop 46 will not move in either direction, even if bumped.

Some embodiments include the ability to lock and hold the drill bit 42in a set position relative to the tube 50 in which it is housed. Forexample, in some embodiments, the drill bit 42 can be locked by lockingthe drill stop 46 relative to the tube 50 using a locking mechanism.FIGS. 17F-J illustrates tools for locking and holding a drill bit 42 ina set position in accordance with one embodiment of the invention. Insome embodiments, the locking mechanism 49 shown in FIG. 17H cancomprise two clam shells 49 (shown in FIG. 17F). In some embodiments, adrill bit 42 can be locked into position by assembling the clam shellsaround the drill stop 46 (shown in FIG. 17G). This feature allows theuser to lock the drill bit 42 in a position such that the tip slightlyprotrudes past the end of the tube 50 (see FIGS. 171 and 17J). In thisposition, the user can force the tube 50 to penetrate through softtissues to force the tube 50 to contact bone (for example during apercutaneous spine screw insertion).

In some further embodiments, the end-effectuator 30 can be configurednot block the tracking optical markers 720 or interfere with thesurgeon. For example, in some embodiments, the end-effectuator 30 cancomprise a clearance mechanism 33 including an actuator 33 a thatpermits this configuration, as depicted in FIGS. 18A and 18B. As shown,the guide tube 50 can be secured within a housing of the end-effectuator30 with two shafts 32. In some embodiments, the shafts 32 move relativeto one other, due to a parallelogram effect of the clearance mechanism33, the position of the guide tube 50 can mimic the position of theend-effectuator 30 (see FIG. 18B).

In applications such as cervical or lumbar fusion surgery, it can bebeneficial to apply distraction or compression across one or more levelsof the spine (anteriorly or posteriorly) before locking hardware inplace. In some embodiments, the end-effectuator 30 can comprise anattachment element 37 that is configured to apply such forces (see forexample FIGS. 19A-B). In some embodiments, the end-effectuator 30attachment element 37 can be configured for coupling to theend-effectuator 30 at substantially the same location as the clearancemechanism 33. In some embodiments, the end-effectuator 30 withattachment element 37 snaps into the same place as the end-effectuator30 without the attachment element 37. In some embodiments, during use ofthe end-effectuator 30 attachment element 37, the relative movement ofthe two shafts 32 caused by angulation 30 e will not cause movement inthe pitch 60 direction and will instead cause distraction (illustratedas moving from an attachment element 37 distance 37 a in FIG. 19A todistance 37 b in FIG. 19B). Further, although shaft 32 movement as shownin FIGS. 19A-B would cause distraction, rotation of the actuator 33 a inthe opposite direction to that represented by 30 e would causecompression (i.e. the distance 37 b in FIG. 19B would move towards thedistance 37 a in FIG. 19A).

In view of the embodiments described hereinbefore, some embodiments thatcan be implemented in accordance with the disclosed subject matter canbe better appreciated with reference to the flowcharts in FIGS. 24-33.For purposes of simplicity of explanation, the method disclosed by theembodiments described herein is presented and described as a series ofsteps; however, it is to be understood and appreciated that the claimedsubject matter is not limited by the order of acts, as some acts mayoccur in different orders and/or concurrently with other acts from thatshown and described herein. For example, the various methods orprocesses of some embodiments of the invention can alternatively berepresented as a series of interrelated states or events, such as in astate diagram. Furthermore, not all illustrated acts may be required toimplement a method in accordance with some embodiments of the invention.Further yet, two or more of the disclosed methods or processes can beimplemented in combination with each other, to accomplish one or morefeatures or advantages herein described.

It should be further appreciated that the methods disclosed in thevarious embodiments described throughout the subject specification canbe stored on an article of manufacture, or computer-readable medium, tofacilitate transporting and transferring such methods to a computingdevice (e.g., a desktop computer, a mobile computer, a mobile telephone,a blade computer, a programmable logic controller, and the like) forexecution, and thus implementation, by a processor of the computingdevice or for storage in a memory thereof.

In some embodiments, the surgical robot 15 can adjust its positionautomatically continuously or substantially continuously in order tomove the end-effectuator 30 to an intended (i.e. planned) position. Forexample, in some embodiments, the surgical robot 15 can adjust itsposition automatically continuously or substantially continuously basedon the current position of the end-effectuator 30 and surgical target asprovided by a current snapshot of tracking markers, LPS, or othertracking data. It should further be appreciated that certain positionadjustment strategies can be inefficient. For example, an inefficientstrategy for the robot 15 to find a target location can be an iterativealgorithm to estimate the necessary direction of movement, move towardthe target location, and then assess a mismatch between a currentlocation and the target location (the mismatch referred to as an error),and estimate a new direction, repeating the cycle ofestimate-movement-assessment until the target location is reached withina satisfactory error. Conversely, the position adjustment strategies inaccordance with some embodiments of the invention are substantively moreefficient than iterative strategies. For example, in some embodiments, asurgical robot 15 can make movements and adjust its location bycalibrating the relative directions of motions in each axis (permittingcomputation via execution of software or firmware with the computer 100)at each frame of tracking data, of a unique set of necessary motorencoder counts that can cause each of the individual axes to move to thecorrect location. In some embodiments, the Cartesian design of thedisclosed robot 15 can permit such a calibration to be made byestablishing a coordinate system for the robot 15 and determining keyaxes of rotation.

As described in greater detail below, in some embodiments, methods forcalibrating the relative directions of the robot's 15 axes can utilize asequence of carefully planned movements, each in a single axis. In someembodiments, during these moves, temporary tracking markers 720 areattached to the end-effectuator 30 to capture the motion of theend-effectuator 30. It should be appreciated that the disclosed methodsdo not require the axes of the robot 15 to be exactly or substantiallyperpendicular, nor do they require the vector along which a particularaxis moves (such as the x-axis 66) to coincide with the vector aboutwhich rotation occurs (such as pitch 60, which occurs primarily aboutthe x-axis 66). In certain embodiments, the disclosed methods includemotion along a specific robot 15 axis that occurs in a straight line. Insome embodiments, the disclosed methods for calibrating the relativedirections of movement of the robot's 15 axes can utilize one or moreframes of tracking data captured at the ends of individual moves made inx-(66), y-(68), roll (62), pitch (60), and Z-tube axes 64 from markers720 temporarily attached to the end-effectuator's 30 guide tube 50. Insome embodiments, when moving individual axes, all other axes can beconfigured at the zero position (for example, the position where theencoder for the axis reads 0 counts). Additionally or alternatively, oneor more frames of tracking data with all robot 15 axes at 0 counts(neutral position) may be necessary, and one or more frames of data withthe temporary markers 720 rotated to a different position about thelongitudinal axis of the guide tube 50 may be necessary. In someembodiments, the marker 720 positions from these moves can be used toestablish a Cartesian coordinate system for the robot 15 in which theorigin (0, 0, 0) is through the center of the end-effectuator 30 and isat the location along the end-effectuator 30 closest to where pitch 60occurs. Additionally or alternatively, in some embodiments, thiscoordinate system can be rotated to an alignment in which y-axis 68movement of the robot 15 can occur exactly or substantially along thecoordinate system's y-axis 68, while x-axis 66 movement of the robot 15occurs substantially perpendicular to the y-axis 68, but by constructionof the coordinate system, without resulting in any change in the z-axis70 coordinate. In certain embodiments, the steps for establishing therobot's 15 coordinate system based at least on the foregoing individualmoves can comprise the following: First, from the initial and finalpositions of the manual rotation of tracking markers 720 about the longaxis of the end-effectuator 30, a finite helical axis of motion iscalculated, which can be represented by a vector that is centered in andaligned with the end-effectuator 30. It should be appreciated thatmethods for calculating a finite helical axis of motion from twopositions of three or more markers are described in the literature, forexample, by Spoor and Veldpaus (Spoor, C. W. and F. E. Veldpaus, “Rigidbody motion calculated from spatial co-ordinates of markers,” J Biomech13(4): 391-393 (1980)). In some embodiments, rather than calculating thehelical axis, the vector that is centered in and aligned with theend-effectuator 30 can be defined, or constructed, by interconnectingtwo points that are attached to two separate rigid bodies that can betemporarily affixed to the entry and exit of the guide tube 50 on theZ-tube axis 64. In this instance, each of the two rigid bodies caninclude at least one tracking marker 720 (e.g., one tracking marker 720,two tracking markers 720, three tracking markers 720, more than threetracking markers 720, etc.), and a calibration can be performed thatprovides information indicative of the locations on the rigid bodiesthat are adjacent to the entry and exit of the guide tube 50 relative tothe tracking markers.

A second helical axis can be calculated from the pitch 60 movements,providing a vector substantially parallel to the x-axis of the robot 15but also close to perpendicular with the first helical axis calculated.In some embodiments, the closest point on the first helical axis to thesecond helical axis (or vector aligned with the end-effectuator 30) iscalculated using simple geometry and used to define the origin of therobot's coordinate system (0, 0, 0). A third helical axis is calculatedfrom the two positions of the roll 62 axis. In certain scenarios, itcannot be assumed that the vector about which roll occurs (third helicalaxis) and the vector along which the y-axis 68 moves are exactly orsubstantially parallel. Moreover, it cannot be assumed that the vectorabout which pitch 60 occurs and the vector along which x-axis 66 motionoccurs are exactly or substantially parallel. Vectors for x-axis 66 andy-axis 68 motion can be determined from neutral and extended positionsof x-axis 66 and y-axis 68 and stored separately. As described herein,in some embodiments, the coordinate system can be realigned to enabley-axis movement of the robot 15 to occur exactly or substantially in they-axis 68 direction of the coordinate system, and x-axis 66 movement ofthe robot 15 without any change in the z-coordinate (70). In general, toperform such a transformation of coordinate systems, a series ofrotations about a coordinate axis is performed and applied to everypoint of interest in the current coordinate system. Each point is thenconsidered to be represented in the new coordinate system. In someembodiments, to apply a rotation of a point represented by a 3×1 vectorabout a particular axis, the vector can be pre-multiplied by a 3×3rotation matrix. The 3×3 rotation matrix for a rotation of Rx degreesabout the x-axis is:

$\quad\begin{bmatrix}1 & 0 & 0 \\0 & {\cos \; R_{x}} & {{- \sin}\; R_{x}} \\0 & {\sin \; R_{x}} & {\cos \; R_{x}}\end{bmatrix}$

The 3×3 rotation matrix for a rotation of R_(y) degrees about the y-axisis:

$\quad\begin{bmatrix}{\cos \; R_{y}} & 0 & {\sin \; R_{y}} \\0 & 1 & 0 \\{{- \sin}\; R_{y}} & 0 & {\cos \; R_{y}}\end{bmatrix}$

The 3×3 rotation matrix for a rotation of R_(z) degrees about the z-axisis:

$\quad\begin{bmatrix}{\cos \; R_{z}} & {{- \sin}\; R_{z}} & 0 \\{\sin \; R_{z}} & {\cos \; R_{z}} & 0 \\0 & 0 & 1\end{bmatrix}$

In some embodiments, to transform coordinate systems, a series of threerotations can be performed. For example, such rotations can be appliedto all vectors and points of interest in the current coordinate system,including the x-movement vector, y-movement vector and each of thehelical axe, to align the y movement vector with the new coordinatesystem's y-axis, and to align the x movement vector as closely aspossible to the new coordinate system's x-axis at z=0. It should beappreciated that more than one possible sequence of three rotations canbe performed to achieve substantially the same goal. For example, insome embodiments, a sequence of three rotations can comprise (1) arotation about x using an R_(x) value appropriate to rotate they-movement vector until its z coordinate equal 0, followed by (2) arotation about z using an R_(z) value appropriate to rotate they-movement vector until its x coordinate equal 0, followed by (3) arotation about y using an R_(y) value appropriate to rotate thex-movement vector until its z coordinate equals 0. In some embodiments,to find the rotation angle appropriate to achieve a given rotation, thearctangent function can be utilized. For example, in some embodiments,the angle needed to rotate a point or vector (x1, y1, z1) about the zaxis to y1=0 is −arctan(y1/x1).

It should be appreciated that after transformation of the coordinatesystem, in some embodiments, although the new coordinate system isaligned such that the y-movement axis of the surgical robot 15 isexactly or substantially exactly aligned with the coordinate system'sy-axis 68, the roll 62 rotation movement of the robot 15 should not beassumed to occur exactly or substantially exactly about a vector alignedwith the coordinate system's y-axis 68. Similarly, in some embodiments,the pitch 60 movement of the surgical robot 15 should not be assumed tooccur exactly or substantially exactly about a vector aligned with thecoordinate system's x-axis. In some embodiments, in roll 62 and pitch 60rotational movement there can be linear and orientational “offsets” fromthe helical axis of motion to the nearest coordinate axis. In someembodiments, from the helical axes determined above using trackedmarkers, such offsets can be calculated and retained (e.g., stored in acomputing device's memory) so that for any rotation occurring duringoperation, the offsets can be applied, rotation can be performed, andthen negative offsets can be applied so that positional change occurringwith rotation motion accounts for the true center of rotation.

In some embodiments, during tracking, the desired trajectory can befirst calculated in the medical image coordinate system, thentransformed to the robot 15 coordinate system based at least on knownrelative locations of active markers. For example, in some embodiments,conventional light-emitting markers and/or conventional reflectivemarkers associated with an optical tracking system 3417 can be used (seefor example active markers 720 in FIG. 20A). In other embodiments,conventional electromagnetic sensors associated with an electromagnetictracking system can be used. In some other embodiments, radio-opaquemarkers (for example markers 730 shown in FIG. 20A) can be used with aCT imaging system. In some embodiments, radio-opaque markers 730(spheres formed, at least in part from metal or other dense material),can be used to provide a marker 730 that can at least partially absorbx-rays to produce a highly contrasted image of the sphere in a CT scanimage.

In some embodiments, the necessary counts for the end-effectuator 30 toreach the desired position in the robot's 15 coordinate system can becalculated based on the following example process. First the necessarycounts to reach the desired angular orientation can be calculated. Insome embodiments, a series of three rotations can be applied to shiftthe coordinate system temporarily to a new coordinate system in whichthe y-axis 68 coincides or substantially coincides with the helical axisof motion for roll 62, and the x-axis 66 is largely aligned with thehelical axis of motion for pitch 60 and by definition, and the helicalaxis of motion for pitch 60 has constant z=0. Then, the number of countsnecessary to achieve the desired pitch 60 can be determined, keepingtrack of how this pitch 60 can affect roll 62. In one implementation, tofind the necessary counts to achieve the desired pitch, the change inpitch angle 60 can be multiplied by the previously calibrated motorcounts per degree for pitch. The change in roll 62 caused by this changein pitch 60 can be calculated from the orientation of the helical axisand the rotation angle (pitch) about the helical axis. Then, thenecessary roll 62 to get to the desired roll 62 to reach the plannedtrajectory alignment can be calculated, with the benefit that applyingroll 62 does not, by definition of the coordinate system, result in anyfurther change in pitch. The coordinate system is then shifted back tothe previously described robot 15 coordinate system by the inverse ofthe three rotations applied above. Then the necessary counts to reachthe desired x-axis 66 position can be calculated, also keeping track ofhow this x-axis 66 position change will affect y-axis 68 position. Thenthe necessary y-axis 68 counts to reach the desired y-axis position canbe readily calculated with the benefit that changing the y-axis 68coordinate can have no effect on any other axis since the y-axis motionvector is by definition aligned with the robot's y-axis 68. In ascenario in which the Z-tube 50 position is being actively controlled,the orientation of the Z-tube 50 movement vector is adjusted whenadjusting roll 62 and pitch 60 and the counts necessary to move it tothe desired position along the trajectory vector is calculated from theoffset. In some embodiments, after the necessary counts to achieve thedesired positions in all axes are calculated as described, these countscan be sent as computer-accessible instructions (e.g., computer-readableand/or computer-executable instructions) to respective controllers foreach axis in order to move the axes to the computed positions.

FIG. 24 is a flowchart of a method 2400 for positioning and advancingthrough soft tissue in accordance with one or more aspects according toone embodiment of the invention. As shown, in some embodiments, at block2410, a medical image is accessed (e.g., received, retrieved, orotherwise acquired). As described herein, the medical image can be a 3Danatomical image scan including, but not limited to a CT scan, amagnetic resonance imaging scan (hereinafter referred to as an “MRIscan”), an X-ray image, or other anatomical scan. It should beappreciated that any 3D anatomical scan may be utilized with thesurgical robot 15 and is within the scope of the present invention. Insome embodiments, at block 2420, a targeting fixture 690 is calibratedto the medical image. In some embodiments, the calibration can besemi-automated or automated. In some embodiments, at block 2430, dataindicative of an intended trajectory associated with the medical imageis received. In some embodiments, at block 2440, a robot 15 issubstantially maintained on the intended trajectory. In someembodiments, a control platform (for example, platform 3400 shown inFIG. 34) can adjust movement of the robot 15 in order to substantiallymaintain the intended trajectory.

FIGS. 25-26 are flowcharts of methods for calibrating a targetingfixture 690 to a medical image in accordance with one or moreembodiments of the invention. As shown in FIG. 25, in some embodiments,the method 2500 can embody a semi-automated calibration method and canbe implemented (e.g., executed) as part of block 2420 in certainscenarios. In some embodiments, at block 2510, data indicative of amedical image having a representation of a plurality of radio-opaquemarkers (for example radio-opaque markers 730) is received. In oneembodiment, as described herein, such plurality can contain fourradio-opaque markers 730. In some embodiments, at block 2520, ageometrical center for each radio-opaque marker 730 is determined in acoordinate system associated with the medical image. In someembodiments, image thresholding can be utilized to define one or moreedges of each radio-opaque marker 730 and a geometrical center thereof.Thresholding refers to an image processing technique in which pixelintensity within a 2D region can be monitored. For example, the x, ypositions (for instance expressed in mm) of pixels of an intensity thatreach a predetermined value can be retrieved. Stated similarly, thethreshold refers to the transition pixel intensity from light to dark.In some embodiments, on 2D slices of the medical image, the radio-opaquemarker 730 can appear light and the adjacent space (such as tissue orair) can appear dark. In some embodiments, displaying pixels thatsatisfy a thresholding criterion at an intensity encountered at the edgeof a radio-opaque marker can yield a largely circular trace outliningthe marker on the medical image. Since in some embodiments, markers 730can be spherical, a method for finding the center of the marker 730 in a2D view can include firstly restricting the 2D view to a sampling regionwith the high-intensity image of the sphere toward the center of theregion and pixels of lower intensity toward the outer edges of theregion. Secondly, the method can include finding the mean x thresholdposition (e.g., the maximum x coordinate of pixels satisfying thethreshold criterion plus minimum x coordinate of pixels satisfying thethreshold criterion divided by two), and finding the mean y thresholdposition using a similar method. In some embodiments, the center of thesphere can be found by determining 2D centers of slices through the samemarker 730 in two orthogonal views. For example, in some embodiments,the method can include finding mean x and mean y from an xy slice, thenfinding mean x and mean z from an xz slice to get a mean x, y, and zaxis coordinate representing the center of the marker 730. Further, uponor after the mean x, mean y, and mean z are found, new xy and xz slicescan be evaluated again and the maximum and minimum x, y, and z thresholdvalues can be again determined to evaluate the dimensions of thethresholded object in each view. It can be appreciated from this methodthat in some embodiments, a non-spherical object of high intensity, suchas a small process of cortical bone extending away from the side of thespine, may fail to satisfy (1) a condition where there is high intensitynear the middle of the region, but low intensity all around, since theprocess may extend out of the region in one or more directions; or (2) acondition where the dimensions in x, y, and z of the centered object donot match each other (e.g., non-spherical case).

As shown in FIG. 25, in some embodiments, at block 2530, it isascertained if one centered sphere is determined for each radio-opaquemarker 730 for the fixture being calibrated. In some embodiments, whenat least one such sphere is not determined, or identified, the thresholdsetting is adjusted and flow is directed to block 2510. In someembodiments, at block 2540, each centered sphere is mapped to eachradio-opaque marker 730 of the plurality of radio-opaque markers 730. Asshown, in some embodiments, block 2540 can represent a mapping actionwhich, in some embodiments, can comprise implementing a sorting processto establish a specific centered sphere is associated with a specificone of the plurality of radio-opaque markers 730. In some embodiments, aplurality of radio-opaque markers 730 contains four radio-opaque markers730 (represented, for example, as OP1, OP2, OP3, and OP4). In someembodiments, the sorting process can map each one of four centeredmarkers 730 to one of OP1, OP2, OP3, or OP4. In some embodiments, thesorting process can distinguish a specific marker 730 by measuringinter-marker distances from mean positions of the four unidentifiedmarkers 730, and comparing such distances to extant inter-markerdistances (for example, those that are pre-measured and retained inmemory, such as mass storage device 3404) for each marker 730 on amarker fixture. In some embodiments, the opaque markers 730 on thefixture 690 can be placed asymmetrically, each marker 730 can beidentified from a unique set of inter-marker distances corresponding tosuch marker 730. For example, in some embodiments where the sum ofinter-marker distances of one unknown marker 730 relative to the otherthrees markers 730 measured from the medical image is D, a singlephysical marker 730 (one of OP1, OP2, OP3, or OP4) can have a matchinginter-marker distance sum within a specified tolerance (such as ±1 mm)of D. In some embodiments, at block 2550, coordinates of each centeredsphere can be retained (for example in memory of a computer platform3400). As described herein, in some embodiments, such coordinates can beutilized in a process for tracking movement of a robot 15.

Some embodiments include method 2600 (shown as a flowchart in FIG. 26)that can embody an automated calibration method and can be implemented(e.g., executed) as part of block 2420 in certain scenarios. In someembodiments, at block 2605, for a Z position, an x-y grid of test areasquares is created. In some embodiments, each test area square can belarger than the diameter of a sphere (a radio-opaque marker 730)associated with a targeting fixture 690 comprised of material that, whenimaged, appears as opaque. In some embodiments, each test area squarecan be at least partially overlapping with at least one adjacent testarea square. In one embodiment of the invention, a nearly half thesurface of a test area square can overlap with the surface of anadjacent test area square. In some embodiments, at block 2610,calibration is initiated at place Z=0, x-y grid row 0, x-y grid column0. In some embodiments, at block 2615, borders of a medical image withina first test area square are determined. It should be appreciated thatin some embodiments, a sphere can be rendered as a circular area, but asection of bone represented in the medical image can be asymmetrical. Insome embodiments, a thresholding process in accordance with one or moreaspects described herein can be implemented to exclude one or moreinvalid markers 730 by assessing if the x, y, and z axes boundaries ofthe object are of substantially equivalent dimensions, consistent withthe shape being spherical.

In some embodiments, at block 2620, it is determined if a maximum (max)border coordinate is less than the maximum coordinate of the test area,and a minimum (min) border coordinate is greater than the minimumcoordinate of the test area, and vertical span of features rendered inthe image are equal or substantially equal to horizontal span of suchfeatures. As shown in FIG. 26, in some embodiments, in the negativecase, flow is directed to block 2645, at which the first test area ismoved to next grid location and next Z plane. Conversely, in case thethree foregoing conditions are fulfilled, flow is directed to block2625, at which X coordinate and Y coordinate are centered at the centerof the current test area. In some embodiments, at block 2630, Zcoordinate is probed by creating a second test area square spanningupwards and downwards in XZ plane and/or YZ plane to determine one ormore borders of an object. In some embodiments, at block 2635, it isdetermined if borders of the object observed in XZ plane are ofsubstantially equivalent relative spacing (vertically and horizontally)to borders in the x-y plane, consistent with the shape of the objectbeing spherical. In some embodiments, when such borders are of differentspacing, flow is directed to block 2645. Conversely, when spacing ofsuch borders is substantially equivalent between views, a sphere havinga center at X coordinate, Y coordinate, and Z coordinate is identifiedat block 2640 and flow is directed to block 2645.

In some embodiments, at block 2650, it is determined if last row andcolumn in x-y grid are reached and last Z plane is reached as a resultof updating the first test area at block 2645. In some embodiments, inthe negative case, flow is directed to block 2615, in which the firstarea is the updated instance of a prior first area, with the flowreiterating one or more of blocks 2620 through 2645. Conversely, in theaffirmative case, flow is directed to block 2655 at which invalidmarker(s) 730 can be excluded. In some embodiments, a paring process canbe implemented to exclude one or more invalid markers 730. For thisparing process, in some embodiments, the known spacings between each ofthe N radio-opaque markers 730 (with N a natural number) on thetargeting fixture 690 and each other radio-opaque marker 730 on thetargeting fixture 690 can be compared to the markers 730 that have beenfound on the medical image. In a scenario in which more than N number ofmarkers 730 can be found on the medical image, any sphere found on themedical image that does not have spacings relative to N−1 other markers730 that are within an acceptable tolerance of known spacings retained,for example, on a list can be considered to be invalid. For example, ifa targeting fixture 690 has four radio-opaque markers 730, there are sixknown spacings, with each marker 730 having a quantifiable spacingrelative to three other markers 730: the inter-marker spacings formarkers 1-2, 1-3, 1-4, 2-3, 2-4, and 3-4. On the 3D medical image of thetargeting fixture 690, in some embodiments, if five potential markers730 are found on the medical image, their inter-marker spacings can becalculated. In this scenario, there are 10 inter-marker spacings: 1-2,1-3, 1-4, 1-5, 2-3, 2-4, 2-5, 3-4, 3-5, and 4-5, with each sphere havinga quantifiable spacing relative to four other markers 730. Consideringeach of the five potential markers 730 individually, if any one of suchfive markers 730 does not have three of its four inter-marker spacingswithin a very small distance of the spacings on the list of sixpreviously quantified known spacings, it is considered invalid.

In some embodiments, at block 2660, each centered radio-opaque marker730, identified at block 2640, can be mapped to each radio-opaque marker730 of a plurality of radio-opaque markers 730. In some embodiments, asorting process in accordance with one or more aspects described hereincan be implemented to map such markers 730 to radio opaque markers 730.In some embodiments, at block 2665, coordinates of each centered spherecan be retained (e.g., in memory of a computer platform 3400). Asdescribed herein, in some embodiments, such coordinates can be utilizedin a process for tracking movement of a robot 15. In some embodiments,during tracking, the established (e.g., calibrated) spatial relationshipbetween active markers 720 and radio-opaque markers 730 can be utilizedto transform the coordinate system from the coordinate system of themedical image to the coordinate system of the tracking system 3417, orvice versa. Some embodiments include a process for transformingcoordinates from the medical image's coordinate system to the trackingsystem's coordinate system can include a fixture 690 comprising fourradio-opaque markers OP1, OP2, OP3, and OP4 (for example radio-opaquemarkers 730) in a rigidly fixed position relative to four active markersAM1, AM2, AM3, AM4 (for example, active markers 720). In someembodiments, at the time the calibration of the fixture 690 occurred,this positional relationship can be retained in a computer memory (e.g.,system memory 3412) for later access on real-time or substantially onreal-time in a set of four arbitrary reference Cartesian coordinatesystems that can be readily reachable through transformations at anylater frame of data. In some embodiments, each reference coordinatesystem can utilize an unambiguous positioning of three of the activemarkers 720. Some embodiments can include a reference coordinate systemfor AM1, AM2, and AM3 can be coordinate system in which AM1 can bepositioned at the origin (e.g., the three-dimensional vector (0, 0, 0));AM2 can be positioned on the x-axis (e.g., x-coordinate AM2x>0,y-coordinate AM2y=0, and z-coordinate AM2z=0); and AM3 can be positionedon the x-y plane (e.g., x-coordinate AM3x unrestricted, y-coordinateAM3y>0, and z-coordinated AM3z=0). Some embodiments include a method togenerate a transformation to such coordinate system can comprise (1)translation of AM1, AM2, AM3, OP1, OP2, OP3, and OP4 in a manner thatAM1 vector position is (0, 0, 0); (2) rotation about the x-axis by anangle suitable to position AM2 at z=0 (e.g., rotation applied to AM2,AM3 and OP1-OP4); (3) rotation about the z-axis by an angle suitable toposition AM2 at y=0 and x>0 (e.g., rotation applied to AM2, AM3 andOP1-OP4); (4) rotation about the x-axis by an angle suitable to positionAM3 at z=0 and y>0 (e.g., rotation applied to AM3 and OP1-OP4). Itshould be appreciated that, in some embodiments, it is unnecessary toretain these transformations in computer memory, for example; rather,the information retained for later access can be the coordinates ofAM1-AM3 and OP1-OP4 in such reference coordinate system. In someembodiments, another such reference coordinate system can transformOP1-OP4 by utilizing AM2, AM3, and AM4. In some embodiments, anothersuch reference coordinate system can transform OP1-OP4 by utilizing AM1,AM3, and AM4. In some further embodiments, another such referencecoordinate system can transform OP1-OP4 by utilizing AM1, AM2, and AM4.

In some embodiments, at the time of tracking, during any given frame ofdata, the coordinates of the active markers AM1-AM4 can be provided bythe tracking system 3417. In some embodiments, by utilizing markers AM1,AM2, and AM3, transformations suitable to reach the conditions of thereference coordinate system can be applied. In some embodiments, suchtransformations can position AM1, AM2, and AM3 on the x-y plane in aposition in proximity to the position that was earlier stored incomputer memory for this reference coordinate system. In someembodiments, for example, to achieve a best fit of the triad of activemarkers 720 on their stored location, a least squares algorithm can beutilized to apply an offset and rotation to the triad of markers 720. Inone implementation, the least squares algorithm can be implemented asdescribed by Sneath (Sneath P. H. A., Trend-surface analysis oftransformation grids, J. Zoology 151, 65-122 (1967)). In someembodiments, transformations suitable to reach the reference coordinatesystem, including the least squares adjustment, can be retained inmemory (e.g., system memory 3412 and/or mass storage device 3404). Insome embodiments, the retained coordinates of OP1-OP4 in such referencecoordinate system can be retrieved and the inverse of the retainedtransformations to reach the reference coordinate system can be appliedto such coordinates. It should be appreciated that the new coordinatesof OP1-OP4 (the coordinates resulting from application of the inverse ofthe transformations) are in the coordinate system of the tracking system3417. Similarly, in some embodiments, by utilizing the remaining threetriads of active markers 720, the coordinates of OP1-OP4 can beretrieved.

In some embodiments, the four sets of OP1-OP4 coordinates in thetracking system's coordinate system that can be calculated fromdifferent triads of active markers 720 are contemplated to havecoordinates that are approximately equivalent. In some embodiments, whencoordinates are not equivalent, the data set can be analyzed todetermine which of the active markers 720 provides non-suitable (orpoor) data by assessing how accurately each triad of active markers 720at the current frame overlays onto the retained positions of activemarkers 720. In some other embodiments, when the coordinates are nearlyequivalent, a mean value obtained from the four sets can be utilized foreach radio-opaque marker 730. In some embodiments, to transformcoordinates of other data (such as trajectories from the medical imagecoordinate system) to the tracking system's coordinate system, the sametransformations can be applied to the data. For example, in someembodiments, the tip and tail of a trajectory vector can be transformedto the four reference coordinate systems and then retrieved with triadsof active markers 720 at any frame of data and transformed to thetracking system's coordinate system.

FIG. 27 is a flowchart of a method 2700 for automatically maintaining asurgical robot 15 substantially on a trajectory in accordance someembodiments of the invention. In some embodiments, at block 2710, dataindicative of position of one or more of Z-frame 72 or Z-tube 50 arereceived. In some embodiments, at block 2720, data indicative of eachrobot 15 joint in the surgical robot 15, such as encoder counts fromeach axis motor 160, are accessed. In some embodiments, at block 2730, acurrent robot 15 position on a planned trajectory is accessed (theposition being represented in a camera coordinate system or thecoordinate system of other tracking device). In one embodiment, theplanned trajectory can be generated by an operator. For example, theoperator (e.g., a surgeon) can scroll and rotate through the imageslices until the desired anatomy can be viewed on three windowsrepresenting three orthogonal planes (typically sagittal, coronal, andaxial slices). The operator can then draw a line at the desired slopeand location on one window; the line simultaneously is calculated andappears on the other two windows, constrained by the views of thescreens and orientation on the window on which it was drawn.

In another embodiment, a line (e.g., referred to as line t) that isfixed on the image both in angle and position represents the desiredtrajectory; the surgeon has to rotate and scroll the images to alignthis trajectory to the desired location and orientation on the anatomy.At least one advantage of such embodiment is that it can provide a morecomplete, holistic picture of the anatomy in relationship to the desiredtrajectory that may not require the operator to erase and start over ornudge the line after it is drawn, and this process was thereforeadopted. In some embodiments, a planned trajectory can be retained in amemory of a computing device (for example, computing device 3401) thatcontrols the surgical robot 15 or is coupled thereto for use during aspecific procedure. In some embodiments, each planned trajectory can beassociated with a descriptor that can be retained in memory with theplanned trajectory. As an example, the descriptor can be the level andside of the spine where screw insertion is planned.

In another embodiment, the line t that is (fixed on the image both inangle and position representing the desired trajectory) is dictated bythe current position of the robot's end effectuator 30, or by anextrapolation of the end effectuator guide tube 50 if an instrument 35were to extend from it along the same vector. In some embodiments, asthe robot 15 is driven manually out over the patient 18 by activatingmotors 160 controlling individual or combined axes 64, 66, 68, 70, theposition of this extrapolated line (robot's end effectuator 30) isupdated on the medical image, based on markers 720 attached to therobot, conventional encoders showing current position of an axis, or acombination of these registers. In some embodiments, when the desiredtrajectory is reached, that vector's position in the medical imagecoordinate system is stored into the computer memory (for example inmemory of a computer platform 3400) so that later, when recalled, therobot 15 will move automatically in the horizontal plane to intersectwith this vector. In some embodiments, instead of manually driving therobot 15 by activating motors 160, the robot's axes can be put in apassive state. In some embodiments, in the passive state, the markers720 continue to collect data on the robot arm 23 position and encoderson each axis 64, 66, 68, 70 continue to provide information regardingthe position of the axis; therefore the position of an extrapolated linecan be updated on the medical image as the passive robot 15 is draggedinto any orientation and position in the horizontal plane. In someembodiments, when a desired trajectory is reached, the position can bestored into the computer memory. Some embodiments include conventionalsoftware control or a conventional switch activation capable of placingthe robot 15 into an active state to immediately rigidly hold theposition or trajectory, and to begin compensating for movement of thepatient 18.

In some further embodiments, the computing device that implements themethod 2700 or that is coupled to the surgical robot 15 can render oneor more planned trajectories. Such information can permit confirmingthat the trajectories planned are within the range of the robot's 15reach by calculating the necessary motor 160 encoder counts to reacheach desired trajectory, and assessing if the counts are within therange of possible counts of each axis.

In some embodiments, information including whether each trajectory is inrange, and how close each trajectory is to being out of range can beprovided to an agent (such as a surgeon or other user, or equipment).For example, in some embodiments, a display means 29 (such as a displaydevice 3411) can render (i.e. display) the limits of axis counts orlinear or angular positions of one or more axes and the position on eachaxis where each targeted trajectory is currently located.

In another embodiment, the display device 3411 (for example, a display150) can render a view of the horizontal work field as a rectangle withthe robot's x-axis 66 movement and y-axis 68 movement ranges definingthe horizontal and vertical dimensions of the rectangle, respectively.In some embodiments, marks (for example, circles) on the rectangle canrepresent the position of each planned trajectory at the currentphysical location of the robot 15 relative to the patient 18. In anotherembodiment, a 3D Cartesian volume can represent the x-axis 66 movement,y-axis 68 movement and z-axis 70 movement ranges of the robot 15. Insome embodiments, line segments or cylinders rendered in the volume canrepresent the position of each planned trajectory at the currentlocation of the robot 15 relative to the patient 18. Repositioning ofthe robot 15 or a patient 18 is performed at this time to a locationthat is within range of the desired trajectories. In other embodiments,the surgeon can adjust the Z Frame 72 position, which can affect thex-axis 66 range and the y-axis 68 range of trajectories that the robot15 is capable of reaching (for example, converging trajectories requireless x-axis 66 or y-axis reach the lower the robot 15 is in the z-axis70). During this time, simultaneously, a screen shows whether trackingmarkers on the patient 18 and robot 15 are in view of the detectiondevice of the tracking system (for example, optical tracking system 3417shown in FIG. 34 and cameras 8200 in FIG. 81). Repositioning of thecameras 8200, if necessary, is also performed at this time for goodvisibility or optimal detection of tracking sensors.

In some embodiments, at block 2740, orientation of an end-effectuator 30in a robot 15 coordinate system is calculated. In some embodiments, atblock 2750, position of the end-effectuator 30 in the robot 15coordinate system is calculated. In some embodiments, at block 2760, aline t defining the planned trajectory in the robot 15 coordinate systemis determined. In some embodiments, at block 2770, robot 15 position islocked on the planned trajectory at a current Z level. In someembodiments, at block 2780, information indicative of quality of thetrajectory lock can be supplied. In some embodiments, actualcoordinate(s) of the surgical robot 15 can be rendered in conjunctionwith respective coordinate(s) of the planned trajectory. In someembodiments, aural indicia can be provided based on such quality. Forinstance, in some embodiments, a high-frequency and/or high-amplitudenoise can embody aural indicia suitable to represent a low-quality lock.In some alternative embodiments, a brief melody may be repeatedlyplayed, such as the sound associated with successful recognition of aUSB memory device by a computer, to indicate successful lock on theplanned trajectory. In other embodiments, a buzz or other warning noisemay be played if the robot 15 is unable to reach its target due to theaxis being mechanically overpowered, or if the tracking markers 720 areundetectable by cameras 8200 or other marker position sensors.

In some embodiments, at block 2790, it is determined if a surgicalprocedure is finished and, in the affirmative case, the flow terminates.In other embodiments, the flow is directed to block 2710. In someembodiments, the method 2700 can be implemented (i.e., executed) as partof block 2440 in certain scenarios. It should be appreciated that insome embodiments, the method 2700 also can be implemented for any robot15 having at least one feature that enable movement of the robot 15.

FIG. 28A is a flowchart of a method 2800 a for calculating positionand/or orientation of an end-effectuator 30 in a robot 15 according toone at least one embodiment of the invention. In some embodiments, theposition and/or orientation can be calculated based at least onmonitored position of a tracking array 690 mounted on the robot's x-axis66 and monitored counts of encoders on the y-axis 68, roll 62, pitch 60,and Z-tube axis 64 actuators. In some embodiments, position and/ororientation are calculated in a robot 15 coordinate system. In someembodiments, the method 2800 a can embody one or more of blocks 2740 or2750. In some embodiments, at block 2805 a, the current position (i.e.,3D position) of x-axis 66 mounted robot 15 tracking markers is accessed.In some embodiments, at block 2810 a, the current position of thetracking array 690 mounted to the robot 15 is transformed to neutralposition. This is a position that was previously stored and representsthe position of the tracking array 690 when the robot 15 was at zerocounts on each axis between the tracker 690 and the robot 15 base(x-axis 66 and z-axis 70 in this configuration). In some embodiments,the set of transformations (T1) to transform from the current positionto the neutral position can be retained in computer memory (for example,the system memory 3412 and/or mass storage device 3404). In someembodiments, a tip and tail of a line segment representing the vector inline with the end-effectuator 30 can be computed based at least on theprocess described herein. In some embodiments, this process canestablish a robot 15 coordinate system and calibrate the relativeorientations of the axes of movement of the robot 15 where trackingmarkers 720 can be attached temporarily to the end-effectuator 30. Insome embodiments, the vector's position in space can be determined byfinding the finite helical axis of motion of markers 720 manuallyrotated to two positions around the guide tube 50. In some embodiments,the vector's position in space can be determined by connecting a pointlocated at the entry of the guide tube 50 (identified by a temporarilymounted rigid body 690 with tracking markers 720) to a point located atthe exit of the guide tube 50 (identified by a second temporarilymounted rigid body 690 with tracking markers 720).

In some embodiments, the tip of the line segment can be obtained as thepoint along the vector that is closest to the vector representing thehelical axis of motion during pitch. In some embodiments, the tail ofthe line segment can be set an arbitrary distance (for example about 100mm) up the vector aligned with the guide tube 50 and/or first helicalaxis. In some embodiments, the Cartesian coordinates of such tip andtail positions can be transformed to a coordinate system describedherein in which the y-axis 68 movement can coincide with the y-axis 68of the coordinate system, and the x-axis 66 can be aligned such thatx-axis 66 movement can cause the greatest change in direction in thex-axis 66, moderate change in the y-axis 68, and no change in the z-axis70. In some embodiments, these coordinates can be retained in a computermemory (for example system memory 3412) for later retrieval. In someembodiments, at block 2815 a, tip and tail coordinates for neutral areaccessed (i.e., retrieved). In some embodiments, at block 2820 a, tipand tail are translated along Z-tube 50 neutral unit vector by monitoredZ-tube 50 counts. In some embodiments, at block 2825 a, an instantaneousaxis of rotation (“IAR”) is accessed. The IAR is the same as the helicalaxis of motion ignoring the element of translation along the helicalaxis for pitch 60 for neutral. As described earlier, in someembodiments, the vectors for this IAR were previously stored in computermemory at the time the coordinate system of the robot 15 was calibrated.In some embodiments, at block 2830 a, tip coordinate, tail coordinate,and IAR vector direction and location coordinates are transformed (forexample, iteratively transformed) to a new coordinate system in whichIAR is aligned with X axis. In some embodiments, data indicative of suchtransformations (T2) can be stored. In some embodiments, at block 2835a, tip coordinate and tail coordinate are rotated about X axis by pitch60 angle. In some embodiments, at block 2840 a, tip coordinate and tailcoordinate are transformed back by inverse of T2 to the previouscoordinate system. In some embodiments, at block 2845 a, previouslystored vectors that represent the IAR for roll 62 are accessed. In someembodiments, at block 2850 a, tip coordinate, tail coordinate, IARcoordinate are transformed (for example, iteratively transformed) to anew coordinate system in which IAR is aligned with y-axis 68. In someembodiments, data indicative of such transformation(s) (T3) can beretained in memory. In some embodiments, at block 2855 a, tip coordinateand tail coordinate are rotated about y-axis 68 by roll 62 angle. Insome embodiments, at block 2860 a, tip coordinate and tail coordinateare transformed back by inverse of T3 to the previous coordinate system.In some embodiments, at block 2865 a, tip coordinate and tail coordinateare translated along a y-axis 68 unit vector (e.g., a vector aligned inthis coordinate system with the y-axis 68) by monitored counts. In someembodiments, at block 2870 a, tip coordinate and tail coordinate aretransformed back by inverse of T1 to the current coordinate systemmonitored by the tracking system 3417.

FIG. 28B is a flowchart of a method 2800 b for calculating positionand/or orientation of an end-effectuator 30 in a robot 15 in accordancewith one embodiment of the invention. In some embodiments, the positionand/or the orientation can be calculated based at least on monitoredposition of a tracking array 690 mounted on the robot's 15 roll 62 axisand monitored counts of encoders on the pitch 60 and Z-tube 50actuators. In some embodiments, position and/or orientation can becalculated in a robot 15 coordinate system. In accordance with someembodiments of the invention, the method 2800B can embody one or more ofblocks 2740 or 2750. In some embodiments, at block 2805 b, currentposition of an array of one or more robot 15 tracking markers 720 isaccessed. In some embodiments, the current position is a 3D position andthe array of robot 15 tracking markers 720 can be mounted to the roll 62axis of the robot 15. In some embodiments, at block 2810 b, the currentposition of the array of robot 15 tracking markers 720 mounted to therobot 15 is transformed to neutral position. In some embodiments, theneutral position can be a position that was previously stored and canrepresent the position of the robot 15 tracking array 690 when the robot15 had zero counts on each axis between the tracker 690 and the robotbase 25 (e.g., z-axis 70, x-axis 68, y-axis 66, and roll 62 axis in thisconfiguration). In some embodiments, data indicative of a set oftransformations (T1) to go from the current position to the neutralposition can be stored in a computer memory (for example, mass storagedevice 3404 or system memory 3412).

In some embodiments, in order to establish a robot 15 coordinate systemand calibrate the relative orientations of the axes of movement of therobot 15, a tip and tail of a line segment representing the vector inline with the end-effectuator 30 with temporarily attached trackingmarkers 720 is located. In some embodiments, the vector's position inspace can be determined by finding the finite helical axis of motion ofmarkers manually rotated to two positions around the guide tube 50. Inother embodiments, the vector's position in space can be determined byconnecting a point located at the entry of the guide tube 50 (identifiedby a temporarily mounted rigid body 690 with tracking markers 720) to apoint located at the exit of the guide tube 50 (identified by a secondtemporarily mounted rigid body 690 with tracking markers 720).

In some embodiments, the tip of the line segment can be found as thepoint along the vector that is closest to the vector representing thehelical axis of motion during pitch. In some embodiments, the tail ofthe line segment can be set an arbitrary distance (for example, nearly100 mm) up the vector aligned with the guide tube/first helical axis. Insome embodiments, the Cartesian coordinates of these tip and tailpositions can be transformed to a coordinate system described herein inwhich the y-axis 68 movement substantially coincides with the y-axis 68of the coordinate system, and the x-axis 66 movement is aligned in amanner that, in some embodiments, x-axis 66 movement causes the greatestchange in direction in the x-axis 66, slight change in y-axis 68, and nochange in the z-axis 70. It should be appreciated that such coordinatescan be retained in memory (for example system memory 3412) for laterretrieval. In some embodiments, at block 2815 b, tip and tailcoordinates for the neutral position are accessed (i.e., retrieved orotherwise obtained). In some embodiments, at block 2820 b, tip and tailare translated along Z-tube 50 neutral unit vector by monitored Z-tube50 counts. In some embodiments, at block 2825 b, IAR is accessed. In oneimplementation, the vectors for this IAR may be available in a computermemory, for example, such vectors may be retained in the computer memoryat the time the coordinate system of the robot 15 is calibrated inaccordance with one or more embodiments described herein. In someembodiments, at block 2830 b, tip coordinate, tail coordinate, and IARvector direction and location coordinates are transformed to a newcoordinate system in which IAR is aligned with x-axis 66. In someembodiments, data indicative of the applied transformations (T2) can beretained in a computer memory. In some embodiments, at block 2835 b, tipcoordinate and tail coordinate are rotated about x-axis 66 by pitch 60angle. In some embodiments, at block 2840 b, tip coordinate and tailcoordinate are transformed back by applying the inverse of T2 to theprevious coordinate system. In some embodiments, at block 2870 b, tipcoordinate and tail coordinate are transformed back by applying theinverse of T1 to the current coordinate system monitored by the trackingsystem 3417.

FIG. 29 is a flowchart of a method 2900 for determining a lineindicative of a trajectory in a robot 15 coordinate system in accordancewith one embodiment of the invention. In some embodiments, thetrajectory can be a planned trajectory associated with a surgicalprocedure. In some embodiments, at block 2910, for a set of currentactive marker 720 positions on a targeting fixture 690, respectiveopaque marker 730 positions are accessed from a rigid body source (suchas a fixture 690). In some embodiments, at block 2920, an opaque marker730 position is transformed from a representation in an image coordinatesystem to a representation in a current active marker 720 coordinatesystem. In some embodiments, at block 2930, a planned trajectory istransformed from a representation in the image coordinate system to arepresentation in the current active maker 720 coordinate system. Insome embodiments, at block 2940, a set of current marker 720 positionson a robot 15 is transformed to a representation in a robot 15coordinate system. In some embodiments, at block 2950, the plannedtrajectory is transformed from a representation in the current activemarker 720 coordinate system to the robot 15 coordinate system.

FIG. 30 is a flowchart of a method 3000 for adjusting a robot 15position to lock on a trajectory in accordance in accordance with oneembodiment of the invention. As illustrated, in some embodiments, thetrajectory can be locked at a current Z plane, or level above thesurgical field 17. In some embodiments, at block 3005, it is determinedif roll 62 of an end-effectuator 30 matches roll of the trajectory(represented by a line t (or t)). In the negative case, in someembodiments, an instruction to move a roll 62 axis is transmitted atblock 3010 and flow is directed to block 3015. Conversely, in theaffirmative case, in some embodiments, flow is directed to block 3015where it is determined if the pitch 60 of the end-effectuator 30 hasmatched the pitch of the trajectory. In the negative case, aninstruction to move a pitch 60 axis is transmitted at block 3020 andflow is directed to block 3025. In the affirmative case, in someembodiments, flow is directed to block 3025 where it is determined ifx-axis 66 coordinates of points on the vector of the end-effectuator 30intercept the x-axis 66 coordinates of the desired trajectory vector. Inthe negative case, in some embodiments, an instruction to move thex-axis 66 can be transmitted and flow is directed to 3035. In theaffirmative case, in some embodiments, flow is directed to block 3035where it is determined if y-axis 68 coordinates of points on the vectorof the end-effectuator 30 intercept the y-axis 68 coordinates of thedesired trajectory vector. In the negative case, in some embodiments, aninstruction to move the y-axis 68 can be transmitted at block 3040 andflow is directed to block 3045. In the affirmative case, in someembodiments, flow is directed to block 3045 in which it is determined ifa Z-tube 50 is being adjusted. In some embodiments, an end-user canconfigure information (i.e., data or metadata) indicative of the Z-tube50 being adjusted to control it to a desired position, for example. Inthe negative case, in some embodiments, flow is terminated. In theaffirmative case, in some embodiments, flow is directed to block 3050where it is determined if the Z-tube 50 is positioned at a predetermineddistance from anatomy. In the affirmative case, in some embodiments,flow terminates and the Z-tube 50 is located at a desired position withrespect to a target location in the anatomy (bone, biopsy site, etc.).In the negative case, in some embodiments, an instruction to move theZ-tube axis 64 is transmitted at block 3055 and the flow is directed toblock 3050. It should be noted that the subject method 3000 in someembodiments, but not all embodiments, may require that movement in eachof the indicated axes (x-axis 66, y-axis 68, Z-tube axis 64, roll 62,and pitch 60) occurs without affecting the other axes earlier in themethod flow. For example, in some embodiments, the y-axis 68 movement atblock 3040 should not cause change in the position of x-axis 66coordinate, which was already checked at block 3025. In someembodiments, the method 3000 can be implemented iteratively in order toreach a desired final position in instances where the axes do not movecompletely independently. In certain embodiments, the method 3000 canaccount for all axis positions nearly simultaneously, and can determinethe exact amount of movement necessary in each axis, and thus it can bemore efficient.

FIG. 31 is a flowchart of a method 3100 for positioning anend-effectuator 30 in space in accordance with one embodiment of theinvention. In some embodiments, the positioning can comprise positioningthe end-effectuator 30 in a first plane (for example, the x-y plane orhorizontal plane) and moving the end-effectuator 30 along a directionsubstantially normal to the first plane. FIGS. 32-33 are flowcharts ofmethods for driving an end-effectuator 30 to a procedure location inaccordance with one embodiment of the invention. As an example, in someembodiments, the procedure location can be a position at the surface ofa bone into which a conventional screw of other piece of hardware is tobe inserted. In some embodiments, the end-effectuator 30 can be fittedwith a guide tube 50 or conventional dilator. In some embodiments, inscenarios in which a Z-tube 50 of a surgical robot 15 comprising theend-effectuator 30 is to be locked and a Z-frame 72 is to be advanced,the method 3200 can be implemented (i.e., executed). In applicationswhere conventional screws are to be driven into bone, the surgeon maywant to move the end-effectuator tip 30, fitted with a guide tube 50 ora conventional dilator, all the way down to the bone (see for exampleFIG. 62 described below). It should be appreciated that in someembodiments, since the first lateral movement occurs above the levelwhere the patient 18 is lying, the methods depicted in FIGS. 31-33 and62 can mitigate the likelihood that the robot 15 randomly collides witha patient 18. In some embodiments, the method can also utilize therobot's Cartesian architecture, and the ease with which a coordinatedmovement down the infinite trajectory vector can be made. That is, insome embodiments, to move down this vector, the roll 62 and pitch 60axes need no adjustment, while the x-axis 66, y-axis 68, and Z-frame 72axes are moved at a fixed rate. In certain embodiments, for an articularrobot 15 to make such a move, the multiple angular axes would have to besynchronized nonlinearly, with all axes simultaneously moved at varyingrates.

FIG. 34 illustrates a block diagram of a computer platform 3400 having acomputing device 3401 that enables various features of the invention,and performance of the various methods disclosed herein in accordancewith some embodiments of the invention. In some embodiments, thecomputing device 3401 can control operation of a surgical robot 15 andan optical tracking system 3417 in accordance with aspects describedherein. In some embodiments, control can comprise calibration ofrelative systems of coordinates, generation of planned trajectories,monitoring of position of various units of the surgical robots 15 and/orunits functionally coupled thereto, and implementation of safetyprotocols, and the like. For example, in some embodiments, computingdevice 3401 can embody a programmable controller that can controloperation of a surgical robot 15 as described herein. It should beappreciated that in accordance with some embodiments of the invention,the operating environment 3400 is only an example of an operatingenvironment and is not intended to suggest any limitation as to thescope of use or functionality of operating environment architecture. Insome embodiments of the invention, the operating environment 3400 shouldnot be interpreted as having any dependency or requirement relating toany one functional element or combination of functional elements (e.g.,units, components, adapters, or the like).

The various embodiments of the invention can be operational withnumerous other general purpose or special purpose computing systemenvironments or configurations. Examples of well-known computingsystems, environments, and/or configurations that can be suitable foruse with the systems and methods of the invention comprise personalcomputers, server computers, laptop devices or handheld devices, andmultiprocessor systems. Additional examples comprise mobile devices,programmable consumer electronics, network PCs, minicomputers, mainframecomputers, distributed computing environments that comprise any of theabove systems or devices, and the like.

In some embodiments, the processing effected in the disclosed systemsand methods can be performed by software components. In someembodiments, the disclosed systems and methods can be described in thegeneral context of computer-executable instructions, such as programmodules, being executed by one or more computers, such as computingdevice 3401, or other computing devices. Generally, program modulescomprise computer code, routines, programs, objects, components, datastructures, etc., that perform particular tasks or implement particularabstract data types. The disclosed methods also can be practiced ingrid-based and distributed computing environments where tasks areperformed by remote processing devices that are linked through acommunications network. In a distributed computing environment, programmodules can be located in both local and remote computer storage mediaincluding memory storage devices.

Further, one skilled in the art will appreciate that the systems andmethods disclosed herein can be implemented via a general-purposecomputing device in the form of the computing device 3401. In someembodiments, the components of the computing device 3401 can comprise,but are not limited to, one or more processors 3403, or processing units3403, a system memory 3412, and a system bus 3413 that couples varioussystem components including the processor 3403 to the system memory3412. In some embodiments, in the case of multiple processing units3403, the system can utilize parallel computing.

In general, a processor 3403 or a processing unit 3403 refers to anycomputing processing unit or processing device comprising, but notlimited to, single-core processors; single-processors with softwaremultithread execution capability; multi-core processors; multi-coreprocessors with software multithread execution capability; multi-coreprocessors with hardware multithread technology; parallel platforms; andparallel platforms with distributed shared memory. Additionally oralternatively, a processor 3403 or processing unit 3403 can refer to anintegrated circuit, an application specific integrated circuit (ASIC), adigital signal processor (DSP), a field programmable gate array (FPGA),a programmable logic controller (PLC), a complex programmable logicdevice (CPLD), a discrete gate or transistor logic, discrete hardwarecomponents, or any combination thereof designed to perform the functionsdescribed herein. Processors or processing units referred to herein canexploit nano-scale architectures such as, molecular and quantum-dotbased transistors, switches and gates, in order to optimize space usageor enhance performance of the computing devices that can implement thevarious aspects of the subject invention. In some embodiments, processor3403 or processing unit 3403 also can be implemented as a combination ofcomputing processing units.

The system bus 3413 represents one or more of several possible types ofbus structures, including a memory bus or memory controller, aperipheral bus, an accelerated graphics port, and a processor or localbus using any of a variety of bus architectures. By way of example, sucharchitectures can comprise an Industry Standard Architecture (ISA) bus,a Micro Channel Architecture (MCA) bus, an Enhanced ISA (EISA) bus, aVideo Electronics Standards Association (VESA) local bus, an AcceleratedGraphics Port (AGP) bus, and a Peripheral Component Interconnects (PCI),a PCI-Express bus, a Personal Computer Memory Card Industry Association(PCMCIA), Universal Serial Bus (USB) and the like. The bus 3413, and allbuses specified in this specification and annexed drawings also can beimplemented over a wired or wireless network connection and each of thesubsystems, including the processor 3403, a mass storage device 3404, anoperating system 3405, robotic guidance software 3406, robotic guidancedata storage 3407, a network adapter 3408, system memory 3412, aninput/output interface 3410, a display adapter 3409, a display device3411, and a human machine interface 3402, can be contained within one ormore remote computing devices 3414 a,b at physically separate locations,functionally coupled (e.g., communicatively coupled) through buses ofthis form, in effect implementing a fully distributed system.

In some embodiments, robotic guidance software 3406 can configure thecomputing device 3401, or a processor thereof, to perform the automatedcontrol of position of the local robot 3416 (for example, surgical robot15) in accordance with aspects of the invention. Such control can beenabled, at least in part, by a tracking system 3417. In someembodiments, when the computing device 3401 embodies the computer 100functionally coupled to surgical robot 15, robotic guidance software3406 can configure such computer 100 to perform the functionalitydescribed in the subject invention. In some embodiments, roboticguidance software 3406 can be retained in a memory as a group ofcomputer-accessible instructions (for instance, computer-readableinstructions, computer-executable instructions, or computer-readablecomputer-executable instructions). In some embodiments, the group ofcomputer-accessible instructions can encode the methods of the invention(such as the methods illustrated in FIGS. 24-33 in accordance with someembodiments of the invention). In some embodiments, the group ofcomputer-accessible instructions can encode various formalisms (e.g.,image segmentation) for computer vision tracking. Some embodimentsinclude robotic guidance software 3406 that can include a compiledinstance of such computer-accessible instructions, a linked instance ofsuch computer-accessible instructions, a compiled and linked instance ofsuch computer-executable instructions, or an otherwise executableinstance of the group of computer-accessible instructions.

Some embodiments include robotic guidance data storage 3407 that cancomprise various types of data that can permit implementation (e.g.,compilation, linking, execution, and combinations thereof) of therobotic guidance software 3406. In some embodiments, robotic guidancedata storage 3407 can comprise data associated with intraoperativeimaging, automated adjustment of position of the local robot 3416 and/orremote robot 3422, or the like. In some embodiments, the data retainedin the robotic guidance data storage 3407 can be formatted according toany image data in industry standard format. As illustrated, in someembodiments, a remote tracking system 3424 can enable, at least in part,control of the remote robot 3422. In some embodiments, the informationcan comprise tracking information, trajectory information, surgicalprocedure information, safety protocols, and so forth.

In some embodiments of the invention, the computing device 3401typically comprises a variety of computer readable media. The readablemedia can be any available media that is accessible by the computer 3401and comprises, for example and not meant to be limiting, both volatileand non-volatile media, removable and non-removable media. In someembodiments, the system memory 3412 comprises computer readable media inthe form of volatile memory, such as random access memory (RAM), and/ornon-volatile memory, such as read only memory (ROM). In someembodiments, the system memory 3412 typically contains data (such as agroup of tokens employed for code buffers) and/or program modules suchas operating system 3405 and robotic guidance software 3406 that areimmediately accessible to, and/or are presently operated-on by theprocessing unit 3403. In some embodiments, operating system 3405 cancomprise operating systems such as Windows operating system, Unix,Linux, Symbian, Android, Apple iOS operating system, Chromium, andsubstantially any operating system for wireless computing devices ortethered computing devices. Apple® is a trademark of Apple Computer,Inc., registered in the United States and other countries. iOS® is aregistered trademark of Cisco and used under license by Apple Inc.Microsoft® and Windows® are either registered trademarks or trademarksof Microsoft Corporation in the United States and/or other countries.Android® and Chrome® operating system are a registered trademarks ofGoogle Inc. Symbian® is a registered trademark of Symbian Ltd. Linux® isa registered trademark of Linus Torvalds. UNIX® is a registeredtrademark of The Open Group.

In some embodiments, computing device 3401 can comprise otherremovable/non-removable, volatile/non-volatile computer storage media.As illustrated, in some embodiments, computing device 3401 comprises amass storage device 3404 which can provide non-volatile storage ofcomputer code (e.g., computer-executable instructions),computer-readable instructions, data structures, program modules, andother data for the computing device 3401. For instance, in someembodiments, a mass storage device 3404 can be a hard disk, a removablemagnetic disk, a removable optical disk, magnetic cassettes or othermagnetic storage devices, flash memory cards, CD-ROM, digital versatiledisks (DVD) or other optical storage, random access memories (RAM), readonly memories (ROM), electrically erasable programmable read-only memory(EEPROM), and the like.

In some embodiments, optionally, any number of program modules can bestored on the mass storage device 3404, including by way of example, anoperating system 3405, and tracking software 3406. In some embodiments,each of the operating system 3405 and tracking software 3406 (or somecombination thereof) can comprise elements of the programming and thetracking software 3406. In some embodiments, data and code (for example,computer-executable instructions, patient-specific trajectories, andpatient 18 anatomical data) can be retained as part of tracking software3406 and stored on the mass storage device 3404. In some embodiments,tracking software 3406, and related data and code, can be stored in anyof one or more databases known in the art. Examples of such databasescomprise, DB2®, Microsoft® Access, Microsoft® SQL Server, Oracle®,mySQL, PostgreSQL, and the like. Further examples include membasedatabases and flat file databases. The databases can be centralized ordistributed across multiple systems.

DB2® is a registered trademark of IBM in the United States.

Microsoft®, Microsoft® Access®, and Microsoft® SQL Server™ are eitherregistered trademarks or trademarks of Microsoft Corporation in theUnited States and/or other countries.

Oracle® is a registered trademark of Oracle Corporation and/or itsaffiliates.

MySQL® is a registered trademark of MySQL AB in the United States, theEuropean Union and other countries.

PostgreSQL® and the PostgreSQL® logo are trademarks or registeredtrademarks of The PostgreSQL Global Development Group, in the U.S. andother countries.

In some embodiments, an agent (for example, a surgeon or other user, orequipment) can enter commands and information into the computing device3401 via an input device (not shown). Examples of such input devices cancomprise, but are not limited to, a camera (or other detection devicefor non-optical tracking markers), a keyboard, a pointing device (forexample, a mouse), a microphone, a joystick, a scanner (for example, abarcode scanner), a reader device such as a radiofrequencyidentification (RFID) readers or magnetic stripe readers, gesture-basedinput devices such as tactile input devices (for example, touch screens,gloves and other body coverings or wearable devices), speech recognitiondevices, or natural interfaces, and the like. In some embodiments, theseand other input devices can be connected to the processing unit 3403 viaa human machine interface 3402 that is coupled to the system bus 3413.In some other embodiments, they can be connected by other interface andbus structures, such as a parallel port, game port, an IEEE 1394 port(also known as a firewire port), a serial port, or a universal serialbus (USB).

In some further embodiments, a display device 3411 can also befunctionally coupled to the system bus 3413 via an interface, such as adisplay adapter 3409. In some embodiments, the computer 3401 can havemore than one display adapter 3409 and the computer 3401 can have morethan one display device 3411. For example, in some embodiments, adisplay device 3411 can be a monitor, a liquid crystal display, or aprojector. Further, in addition to the display device 3411, someembodiments can include other output peripheral devices that cancomprise components such as speakers (not shown) and a printer (notshown) capable of being connected to the computer 3401 via input/outputInterface 3410. In some embodiments, the input/output interface 3410 canbe a pointing device, either tethered to, or wirelessly coupled to thecomputing device 3410. In some embodiments, any step and/or result ofthe methods can be output in any form to an output device. In someembodiments, the output can be any form of visual representation,including, but not limited to, textual, graphical, animation, audio,tactile, and the like.

In certain embodiments, one or more cameras (for example, camera 8200shown in FIG. 81) can be contained or functionally coupled to thetracking system 3417, which is functionally coupled to the system bus3413 via an input/output interface of the one or more input/outputinterfaces 3410. Such functional coupling can permit the one or morecamera(s) to be coupled to other functional elements of the computingdevice 3401. In one embodiment, the input/output interface, at least aportion of the system bus 3413, and the system memory 3412 can embody aframe grabber unit that can permit receiving imaging data acquired by atleast one of the one or more cameras. In some embodiments, the framegrabber can be an analog frame grabber, a digital frame grabber, or acombination thereof. In some embodiments, where the frame grabber is ananalog frame grabber, the processor 3403 can provide analog-to-digitalconversion functionality and decoder functionality to enable the framegrabber to operate with medical imaging data. Further, in someembodiments, the input/output interface can include circuitry to collectthe analog signal received from at least one camera of the one or morecameras. In some embodiments, in response to execution by processor3403, tracking software 3406 can operate the frame grabber to receiveimaging data in accordance with various aspects described herein.

Some embodiments include a computing device 3401 that can operate in anetworked environment (for example, an industrial environment) usinglogical connections to one or more remote computing devices 3414 a,b, aremote robot 3422, and a tracking system 3424. By way of example, insome embodiments, a remote computing device can be a personal computer,portable computer, a mobile telephone, a server, a router, a networkcomputer, a peer device or other common network node, and so on. Inparticular, in some embodiments, an agent (for example, a surgeon orother user, or equipment) can point to other tracked structures,including anatomy of a patient 18, using a remote computing device 3414such as a hand-held probe that is capable of being tracked andsterilized. In some embodiments, logical connections between thecomputer 3401 and a remote computing device 3414 a,b can be made via alocal area network (LAN) and a general wide area network (WAN). In someembodiments, the network connections can be implemented through anetwork adapter 3408. In some embodiments, the network adapter 3408 canbe implemented in both wired and wireless environments. Some embodimentsinclude networking environments that can be conventional and commonplacein offices, enterprise-wide computer networks, intranets. In someembodiments, the networking environments generally can be embodied inwire-line networks or wireless networks (for example, cellular networks,such as third generation (“3G”) and fourth generation (“4G”) cellularnetworks, facility-based networks (for example, femtocell, picocell,wifi networks). In some embodiments, a group of one or more networks3415 can provide such networking environments. In some embodiments ofthe invention, the one or more network(s) can comprise a LAN deployed inan industrial environment comprising the system 1 described herein.

As an illustration, in some embodiments, application programs and otherexecutable program components such as the operating system 3405 areillustrated herein as discrete blocks, although it is recognized thatsuch programs and components reside at various times in differentstorage components of the computing device 3401, and are executed by thedata processor(s) of the computer 100. Some embodiments include animplementation of tracking software 3406 that can be stored on ortransmitted across some form of computer readable media. Any of thedisclosed methods can be performed by computer readable instructionsembodied on computer readable media. Computer readable media can be anyavailable media that can be accessed by a computer. By way of exampleand not meant to be limiting, computer-readable media can comprise“computer storage media,” or “computer-readable storage media,” and“communications media.” “Computer storage media” comprise volatile andnon-volatile, removable and non-removable media implemented in anymethods or technology for storage of information such as computerreadable instructions, data structures, program modules, or other data.In some embodiments of the invention, computer storage media comprises,but is not limited to, RAM, ROM, EEPROM, flash memory or other memorytechnology, CD-ROM, digital versatile disks (DVD) or other opticalstorage, magnetic cassettes, magnetic tape, magnetic disk storage orother magnetic storage devices, or any other medium which can be used tostore the desired information and which can be accessed by a computer.

As described herein, some embodiments include the computing device 3401that can control operation of local robots 3416 and/or remote robots3422. Within embodiments in which the local robot 3416 or the remoterobot 3422 are surgical robots 15, the computing device 3401 can executerobotic guidance software 3407 to control such robots 3416, 3422, 15. Insome embodiments, the robotic guidance software 3407, in response toexecution, can utilize trajectories (such as, tip and tail coordinates)that can be planned and/or configured remotely or locally. In anadditional or alternative aspect, in response to execution, the roboticguidance software 3407 can implement one or more of the methodsdescribed herein in a local robot's computer or a remote robot'scomputer to cause movement of the remote robot 15 or the local robot 15according to one or more trajectories.

In some embodiments, the computing device 3401 can enable pre-operativeplanning of the surgical procedure.

In some embodiments, the computing device 3401 can permit spatialpositioning and orientation of a surgical tool (for example, instrument35) during intraoperative procedures. In some further embodiments, thecomputing device 3401 can enable open procedures. In some otherembodiments, the computing device 3401 can enable percutaneousprocedures.

In certain embodiments, the computing device 3401 and the roboticguidance software 3407 can embody a 3D tracking system 3417 tosimultaneously monitor the positions of the device and the anatomy ofthe patient 18. In some embodiments, the 3D tracking system 3417 can beconfigured to cast the patient's anatomy and the end-effectuator 30 in acommon coordinate system.

In some embodiments, the computing device 3401 can access (i.e., load)image data from a conventional static storage device. In someembodiments, the computing device 3401 can permit a 3D volumetricrepresentation of patient 18 anatomy to be loaded into memory (forexample, system memory 3412) and displayed (for example, via displaydevice 3411).

In some embodiments, the computing device 3401, in response to executionof the robotic guidance software 3407 can enable navigation through the3D volume representation of a patient's anatomy.

In some embodiments, the computing device 3401 can operate with aconventional power source required to offer the device for sale in thespecified country. A conventional power cable that supplies power can bea sufficient length to access conventional hospital power outlets. Insome embodiments, in the event of a power loss, the computing device3401 can hold the current end-effectuator 30 in a position unless anagent (for example, a surgeon or other user, or equipment) manuallymoves the end-effectuator 30.

In some embodiments, the computing device 3401 can monitor systemphysical condition data. In some embodiments, the computing device 3401can report to an operator (for example, a surgeon) each of the physicalcondition data and indicate an out-of-range value.

In some embodiments, the computing device 3401 can enable entry andstorage of manufacturing calibration values for end-effectuator 30positioning using, for example, the input/output interface 3410.

In some embodiments, the computing device 3401 can enable access tomanufacturing calibration values by an agent (for example, a surgeon orother user, or equipment) authenticated to an appropriate access level.In some embodiments, the data can be retained in robotic guidance datastorage 3407, or can be accessed via network(s) 3415 when the data isretained in a remote computing device 3414 a.

In some embodiments, the computing device 3401 can render (using forexample display device 3411) a technical screen with a subset of theend-effectuator 30 positioning calibration and system health data. Theinformation is only accessible to an agent (for example, a surgeon orother user, or equipment) authenticated to an appropriate level.

In some embodiments, the computing device 3401 can enable fieldcalibration of end-effectuator 30 positioning only by an agent (forexample, a surgeon or other user, or equipment) authenticated to anappropriate access level.

In some embodiments, the computing device 3401 can convey the status oflocal robot 3416, remote robot 3422, and/or other device being locked inposition using a visual or aural alert.

In some further embodiments, the computing device 3401 can include anemergency stop control that upon activation, disables power to thedevice's motors 160 but not to the processor 3403. In some embodiments,the emergency stop control can be accessible by the operator ofcomputing device 3401. In some embodiments, the computing device 3401can monitor the emergency stop status and indicate to the operator thatthe emergency stop has been activated.

In some other embodiments, the computing device 3401 can be operated ina mode that permits manual positioning of the end-effectuator 30.

In some embodiments, the computing device 3401 can boot directly to anapplication representing the robotic guidance software 3406. In someembodiments, computing device 3401 can perform a system check prior toeach use. In scenarios in which the system check fails, the computingdevice 3401 can notify an operator.

In some embodiments, the computing device 3401 can generate an indicatorfor reporting system status.

Some embodiments include the computing device 3401 that can minimize orcan mitigate delays in processing, and in the event of a delay inprocessing, notify an agent (for example, a surgeon or other user, orequipment). For example, in some embodiments, a delay may occur while asystem scan is being performed to assess system status, and consequentlythe computing device 3401 can schedule (for example, generate a processqueue) system scans to occur at low usage times. In some embodiments, asystem clock of the computing device 3401 can be read before and afterkey processes to assess the length of time required to completecomputation of a process. In some embodiments, the actual time tocomplete the process can be compared to the expected time. In someembodiments, if a discrepancy is found to be beyond an acceptabletolerance, the agent can be notified, and/or concurrently runningnon-essential computational tasks can be terminated. In one embodiment,a conventional system clock (not shown) can be part of processor 3403.

In some embodiments, the computing device 3401 can generate a displaythat follows a standardized workflow.

In some embodiments, the computing device 3401 can render or ensure thattext is rendered in a font of sufficient size and contrast to bereadable from an appropriate distance.

In some embodiments, the computing device 3401 can enable an operator tolocate the intended position of a surgical implant or tool.

In some further embodiments, the computing device 3401 can determine therelative position of the end-effectuator 30 to the anatomy of thepatient 18. For example, to at least such end, the computing device 3401can collect data the optical tracking system 3417, and can analyze thedata to generate data indicative of such relative position.

In some embodiments, the computing device 3401 can indicate theend-effectuator 30 position and orientation.

In some embodiments, the computing device 3401 can enable continuouscontrol of end-effectuator 30 position relative to the anatomy of apatient 18.

In some embodiments, the computing device 3401 can enable an agent (forexample, a surgeon or other user, or equipment) to mark the intendedposition of a surgical implant or tool (for example, instrument 35).

In some embodiments, the computing device 3401 can allow the positionand orientation of a conventional hand-held probe (or an instrument 35)to be displayed overlaid on images of the patient's anatomy.

In some embodiments, the computing device 3401 can enable an agent (forexample, a surgeon or other user, or equipment) to position conventionalsurgical screws. In some embodiments, the computing device 3401 canenable selection of the length and diameter of surgical screws by theagent. In yet another aspect, the computing device can ensure that therelative position, size and scale of screws are maintained on thedisplay 3411 when in graphical representation. In some embodiments, thecomputing device 3401 can verify screw path plans against an operationenvelope and reject screw path plans outside this envelope. In stillanother aspect, the computing device 3401 can enable hiding of agraphical screw representation.

In some embodiments, the computing device 3401 can enable a functionthat allows the current view to be stored. In some embodiments, thecomputing device 3401 can enable a view reset function that sets thecurrent view back to a previously stored view.

In some embodiments, the computing device 3401 can enable anauthentication based tiered access system.

In some embodiments, the computing device 3401 can log and store systemactivity. In some embodiments, the computing device 3401 can enableaccess to the system activity log to an agent authorized to anappropriate level.

In some embodiments, the computing device 3401 can enable entry andstorage of patient 18 data.

In some embodiments, the computing device 3401 can enable theappropriate disposition of patient 18 data and/or procedure data. Forexample, in a scenario in which such data are being collected forresearch, the computing device 3401 can implement de-identification ofthe data in order to meet patient 18 privacy requirements. In someembodiments, the de-identification can be implemented in response toexecution of computer-executable instruction(s) retained in memory 3412or any other memory accessible to the computing device 3401. In someembodiments, the de-identification can be performed automatically beforethe patient 18 data and/or procedure data are sent to a repository orany other data storage (including mass storage device 3404, forexample). In some embodiments, indicia (e.g., a dialog box) can berendered (for example, at display device 3411) to prompt an agent (e.g.,machine or human) to permanently delete patient 18 data and/or proceduredata at the end of a procedure.

FIG. 11 shows a flow chart diagram 1100 for general operation of therobot 15 according to some embodiments is shown. In some embodiments, atstep 210, the local positioning system (herein referred to as “LPS”)establishes a spatial coordinate measuring system for the room 10 wherethe invasive procedure is to occur; in other words, the LPS iscalibrated. In some embodiments, in order to calibrate the LPS, aconventional mechanical fixture that includes a plurality of attachedcalibrating transmitters 120 is placed within the room 10 wherepositioning sensors 12 are located. In some embodiments of theinvention, at least three calibrating transmitters 120 are required, butany number of calibrating transmitters 120 above three is within thescope of the invention. Also, in some embodiments, at least threepositioning sensors 12 are required, but any number of positioningsensors 12 above three is also within the scope of the invention, andthe accuracy of the system is increased with the addition of morepositioning sensors.

In some embodiments, the distance between each of the calibratingtransmitters 120 relative to each other is measured prior to calibrationstep 210. Each calibrating transmitter 120 transmits RF signals on adifferent frequency so that the positioning sensors 12 can determinewhich transmitter 120 emitted a particular RF signal. In someembodiments, the signal of each of these transmitters 120 is received bypositioning sensors 12. In some embodiments, since the distance betweeneach of the calibrating transmitters 120 is known, and the sensors 12can identify the signals from each of the calibrating transmitters 120based on the known frequency, using time of flight calculation, thepositioning sensors 12 are able to calculate the spatial distance ofeach of the positioning sensors 12 relative to each other. The system 1is now calibrated. As a result, in some embodiments, the positioningsensors 12 can now determine the spatial position of any new RFtransmitter 120 introduced into the room 10 relative to the positioningsensors 12.

In some embodiments, a step 220 a in which a 3D anatomical image scan,such as a CT scan, is taken of the anatomical target. Any 3D anatomicalimage scan may be used with the surgical robot 15 and is within thescope of the present invention.

In some embodiments, at step 230, the positions of the RF transmitters120 tracking the anatomical target are read by positioning sensors 110.These transmitters 120 identify the initial position of the anatomicaltarget and any changes in position during the procedure.

In some embodiments, if any RF transmitters 120 must transmit through amedium that changes the RF signal characteristics, then the system willcompensate for these changes when determining the transmitter's 120position.

In some embodiments, at step 240, the positions of the transmitters 120on the anatomy are calibrated relative to the LPS coordinate system. Inother words, the LPS provides a reference system, and the location ofthe anatomical target is calculated relative to the LPS coordinates. Insome embodiments, to calibrate the anatomy relative to the LPS, thepositions of transmitters 120 affixed to the anatomical target arerecorded at the same time as positions of temporary transmitters 120placed on precisely known anatomical landmarks also identified on theanatomical image. This calculation is performed by a computer 100.

In some embodiments, at step 250, the positions of the RF transmitters120 that track the anatomical target are read. Since the locations ofthe transmitters 120 on the anatomical target have already beencalibrated, the system can easily determine if there has been any changein position of the anatomical target.

Some embodiments include a step 260, where the positions of thetransmitters 120 on the surgical instrument 35 are read. Thetransmitters 120 may be located on the surgical instrument 35 itself,and/or there may be transmitters 120 attached to various points of thesurgical robot 15.

In some embodiments of the invention, the surgical robot 15 can alsoinclude a plurality of attached conventional position encoders that helpdetermine the position of the surgical instrument 35. In someembodiments, the position encoders can be devices used to generate anelectronic signal that indicates a position or movement relative to areference position. In some other embodiments, a position signal can begenerated using conventional magnetic sensors, conventional capacitivesensors, and conventional optical sensors.

In some embodiments, position data read from the position encoders maybe used to determine the position of the surgical instrument 35 used inthe procedure. In some embodiments, the data may be redundant ofposition data calculated from RF transmitters 120 located on thesurgical instrument 35. Therefore, in some embodiments, position datafrom the position encoders may be used to double-check the positionbeing read from the LPS.

In some embodiments, at step 270, the coordinates of the positions ofthe transmitters 120 on the surgical instrument 35, and/or the positionsread from the position encoders, is calibrated relative to theanatomical coordinate system. In other words, in some embodiments, theposition data of the surgical instrument 35 is synchronized into thesame coordinate system as the patient's anatomy. In some embodiments,this calculation is performed automatically by the computer 100 sincethe positions of the transmitters 120 on the anatomical target and thepositions of the transmitters 120 on the surgical instrument 35 are inthe same coordinate system, and the positions of the transmitters 120 onthe anatomical target are already calibrated relative to the anatomy.

In some embodiments, at step 280, the computer 100 superimposes arepresentation of the location calculated in step 270 of the surgicaldevice on the 3D anatomical image of the patient 18 taken in step 220.In some embodiments, the superimposed image can be displayed to anagent.

In some embodiments, at step 290, the computer 100 sends the appropriatesignals to the motors 160 to drive the surgical robot 15. In someembodiments, if the agent preprogrammed a trajectory, then the robot 15is driven so that the surgical instrument 35 follows the preprogrammedtrajectory if there is no further input from the agent. In someembodiments, if there is agent input, then the computer 100 drives therobot 15 in response to the agent input.

In some embodiments, at step 295, the computer 100 determines whetherthe anatomy needs to be recalibrated. In some embodiments, the agent maychoose to recalibrate the anatomy, in which case the computer 100responds to agent input. Alternatively, in some embodiments, thecomputer 100 may be programmed to recalibrate the anatomy in response tocertain events. For instance, in some embodiments, the computer 100 maybe programmed to recalibrate the anatomy if the RF transmitters 120 onthe anatomical target indicate that the location of the anatomicaltarget has shifted relative to the RF transmitters 120 (i.e. thisspatial relationship should be fixed). In some embodiments, an indicatorthat the anatomical target location has shifted relative to thetransmitters 120 is if the computer 100 calculates that the surgicalinstrument 35 appears to be inside bone when no drilling or penetrationis actually occurring.

In some embodiments, if the anatomy needs to be calibrated, then theprocess beginning at step 230 is repeated. In some embodiments, if theanatomy does not need to be recalibrated, then the process beginning atstep 250 is repeated.

In some embodiments, at any time during the procedure, certain faultconditions may cause the computer 100 to interrupt the program andrespond accordingly. For instance, in some embodiments, if the signalfrom the RF transmitters 120 cannot be read, then the computer 100 maybe programmed to stop the movement of the robot 15, or remove thesurgical instrument 35 from the patient 18. Another example of a faultcondition is if the robot 15 encounters a resistance above apreprogrammed tolerance level.

FIG. 12 shows a flow chart diagram 1200 for a closed screw/needleinsertion procedure according to an embodiment of the invention isshown. In a closed pedicle screw insertion procedure, in someembodiments, the robot 15 holds a guide tube 50 adjacent to the patient18 in the correct angular orientation at the point where a conventionalpedicle screw is to be inserted through the tissue and into the bone ofthe patient 18.

In some embodiments, the distance between each of the calibratingtransmitters 120 relative to each other is measured prior to calibrationstep 300. In some embodiments, each calibrating transmitter 120transmits RF signals on a different frequency so the positioning sensors12 can determine which transmitter 120 emitted a particular RF signal.In some embodiments, the signal of each of these transmitters 120 isreceived by positioning sensors 12. Since the distance between each ofthe calibrating transmitters 120 is known, and the sensors 12 canidentify the signals from each of the calibrating transmitters 120 basedon the known frequency, using time of flight calculation, in someembodiments, the positioning sensors 12 are able to calculate thespatial distance of each of the positioning sensors 12 relative to eachother. The system 1 is now calibrated. As a result, in some embodiments,the positioning sensors 12 can now determine the spatial position of anynew RF transmitter 120 introduced into the room 10 relative to thepositioning sensors 12.

In some embodiments, at step 310, a 3D anatomical image scan, such as aCT scan, is taken of the anatomical target. Any 3D anatomical image scanmay be used with the surgical robot 15 and is within the scope of thepresent invention.

In some embodiments, at step 320, the operator selects a desiredtrajectory and insertion point of the surgical instrument 35 on theanatomical image captured at step 310. In some embodiments, the desiredtrajectory and insertion point is programmed into the computer 100 sothat the robot 15 can drive a guide tube 50 automatically to follow thetrajectory.

In some embodiments, at step 330, the positions of the RF transmitters120 tracking the anatomical target are read by positioning sensors 110.In some embodiments, these transmitters 120 identify the initialposition of the anatomical target and any changes in position during theprocedure.

In some embodiments, if any RF transmitters 120 must transmit through amedium that changes the RF signal characteristics, the system willcompensate for these changes when determining the transmitter's 120position.

In some embodiments, at step 340, the positions of the transmitters 120on the anatomy are calibrated relative to the LPS coordinate system. Inother words, the LPS provides a reference system, and the location ofthe anatomical target is calculated relative to the LPS coordinates. Insome embodiments, to calibrate the anatomy relative to the LPS, thepositions of transmitters 120 affixed to the anatomical target arerecorded at the same time as positions of temporary transmitters 120 onprecisely known anatomical landmarks also identified on the anatomicalimage. This calculation is performed by a computer.

In some embodiments, at step 350, the positions of the RF transmitters120 that track the anatomical target are read. Since the locations ofthe transmitters 120 on the anatomical target have already beencalibrated, in some embodiments, the system can easily determine ifthere has been any change in position of the anatomical target.

In some embodiments, at step 360, the positions of the transmitters 120on the surgical instrument 35 are read. In some embodiments, thetransmitters 120 may be located on the surgical instrument 35, and/orattached to various points of the surgical robot 15.

In some embodiments, at step 370, the coordinates of the positions ofthe transmitters 120 on the surgical instrument 35, and/or the positionsread from the position encoders, are calibrated relative to theanatomical coordinate system. In other words, the position data of thesurgical instrument 35 is synchronized into the same coordinate systemas the anatomy. This calculation is performed automatically by thecomputer 100 since the positions of the transmitters 120 on theanatomical target and the positions of the transmitters 120 on thesurgical instrument 35 are in the same coordinate system and thepositions of the transmitters 120 on the anatomical target are alreadycalibrated relative to the anatomy.

In some embodiments, at step 380, the computer 100 superimposes arepresentation of the location calculated in step 370 of the surgicaldevice on the 3D anatomical image of the patient 18 taken in step 310.The superimposed image can be displayed to the user.

In some embodiments, at step 390, the computer 100 determines whetherthe guide tube 50 is in the correct orientation and position to followthe trajectory planned at step 320. If it is not, then step 393 isreached. If it is in the correct orientation and position to follow thetrajectory, then step 395 is reached.

In some embodiments, at step 393, the computer 100 determines whatadjustments it needs to make in order to make the guide tube 50 followthe preplanned trajectory. The computer 100 sends the appropriatesignals to drive the motors 160 in order to correct the movement of theguide tube.

In some embodiments, at step 395, the computer 100 determines whetherthe procedure has been completed. If the procedure has not beencompleted, then the process beginning at step 350 is repeated.

In some embodiments, at any time during the procedure, certain faultconditions may cause the computer 100 to interrupt the program andrespond accordingly. For instance, if the signal from the RFtransmitters 120 cannot be read, then the computer 100 may be programmedto stop the movement of the robot 15 or lift the guide tube 50 away fromthe patient 18. Another example of a fault condition is if the robot 15encounters a resistance above a preprogrammed tolerance level. Anotherexample of a fault condition is if the RF transmitters 120 on theanatomical target shift so that actual and calculated positions of theanatomy no longer match. One indicator that the anatomical targetlocation has shifted relative to the transmitters 120 is if the computer100 calculates that the surgical instrument 35 appears to be inside bonewhen no drilling or penetration is actually occurring.

In some embodiments, the proper response to each condition may beprogrammed into the system, or a specific response may beuser-initiated. For example, the computer 100 may determine that inresponse to an anatomy shift, the anatomy would have to be recalibrated,and the process beginning at step 330 should be repeated. Alternatively,a fault condition may require the flowchart to repeat from step 300.Another alternative is the user may decide that recalibration from step330 is desired, and initiate that step himself.

Referring now to FIG. 13, a flow chart diagram 1300 for a safe zonesurgical procedure performed using the system described herein is shownin accordance with some embodiments of the invention. In a safe zonesurgical procedure, there is a defined safe zone around the surgicalarea within which the surgical device must stay. The physician manuallycontrols the surgical device that is attached to the end-effectuator 30of the surgical robot 15. If the physician moves the surgical deviceoutside of the safe zone, then the surgical robot 15 stiffens the arm 23so that the physician cannot move the instrument 35 in any directionthat would move the surgical instrument 35 outside the safe zone.

In some embodiments, the distance between each of the calibratingtransmitters 120 relative to each other is measured prior to calibrationstep 400. Each calibrating transmitter 120 transmits RF signals on adifferent frequency so the positioning sensors 12 can determine whichtransmitter 120 emitted a particular RF signal. The signal of each ofthese transmitters 120 is received by positioning sensors 12. Since thedistance between each of the calibrating transmitters 120 is known, andthe sensors 12 can identify the signals from each of the calibratingtransmitters 120 based on the known frequency, the positioning sensors12 are able to calculate, using time of flight calculation, the spatialdistance of each of the positioning sensors 12 relative to each other.The system 1 is now calibrated. As a result, the positioning sensors 12can now determine the spatial position of any new RF transmitter 120introduced into the room 10 relative to the positioning sensors 12.

In some embodiments, at step 410, a 3D anatomical image scan, such as aCT scan, is taken of the anatomical target. Any 3D anatomical image scanmay be used with the surgical robot 15 and is within the scope of thepresent invention.

In some embodiments, at step 420, the operator inputs a desired safezone on the anatomical image taken in step 410. In an embodiment of theinvention, the operator uses an input to the computer 100 to draw a safezone on a CT scan taken of the patient 18 in step 410.

In some embodiments, at step 430, the positions of the RF transmitters120 tracking the anatomical target are read by positioning sensors.These transmitters 120 identify the initial position of the anatomicaltarget and any changes in position during the procedure.

In some embodiments, if any RF transmitters 120 must transmit through amedium that changes the RF signal characteristics, then the system willcompensate for these changes when determining the transmitter's 120position.

In some embodiments, at step 440, the positions of the transmitters 120on the anatomy are calibrated relative to the LPS coordinate system. Inother words, the LPS provides a reference system, and the location ofthe anatomical target is calculated relative to the LPS coordinates. Tocalibrate the anatomy relative to the LPS, the positions of transmitters120 affixed to the anatomical target are recorded at the same time aspositions of temporary transmitters 120 on precisely known landmarks onthe anatomy that can also be identified on the anatomical image. Thiscalculation is performed by a computer 100.

In some embodiments, at step 450, the positions of the RF transmitters120 that track the anatomical target are read. Since the locations ofthe transmitters 120 on the anatomical target have already beencalibrated, the system can easily determine if there has been any changein position of the anatomical target.

In some embodiments, at step 460, the positions of the transmitters 120on the surgical instrument 35 are read. The transmitters 120 may belocated on the surgical instrument 35 itself, and/or there may betransmitters 120 attached to various points of the surgical robot 15.

In some embodiments, at step 470, the coordinates of the positions ofthe transmitters 120 on the surgical instrument 35, and/or the positionsread from the position encoders, are calibrated relative to theanatomical coordinate system. In other words, the position data of thesurgical instrument 35 is synchronized into the same coordinate systemas the anatomy. This calculation is performed automatically by thecomputer 100 since the positions of the transmitters 120 on theanatomical target and the positions of the transmitters 120 on thesurgical instrument 35 are in the same coordinate system and thepositions of the transmitters 120 on the anatomical target are alreadycalibrated relative to the anatomy.

In some embodiments, at step 480, the computer 100 superimposes arepresentation of the location calculated in step 470 of the surgicaldevice on the 3D anatomical image of the patient 18 taken in step 410.In some embodiments, the superimposed image can be displayed to theuser.

In some embodiments, at step 490, the computer 100 determines whetherthe surgical device attached to the end-effectuator 30 of the surgicalrobot 15 is within a specified range of the safe zone boundary (forexample, within 1 millimeter of reaching the safe zone boundary). Insome embodiments, if the end-effectuator 30 is almost to the boundary,then step 493 is reached. In some embodiments, if it is well within thesafe zone boundary, then step 495 is reached.

In some embodiments, at step 493, the computer 100 stiffens the arm ofthe surgical robot 15 in any direction that would allow the user to movethe surgical device closer to the safe zone boundary.

In some embodiments, at step 495, the computer 100 determines whetherthe anatomy needs to be recalibrated. In some embodiments, the user maychoose to recalibrate the anatomy, in which case the computer 100responds to user input. Alternatively, in some embodiments, the computer100 may be programmed to recalibrate the anatomy in response to certainevents. For instance, in some embodiments, the computer 100 may beprogrammed to recalibrate the anatomy if the RF transmitters 120 on theanatomical target indicate that the location of the anatomical targethas shifted relative to the RF transmitters 120 (i.e. this spatialrelationship should be fixed.) In some embodiments, an indicator thatthe anatomical target location has shifted relative to the transmitters120 is if the computer 100 calculates that the surgical instrument 35appears to be inside bone when no drilling or penetration is actuallyoccurring.

In some embodiments, if the anatomy needs to be calibrated, then theprocess beginning at step 430 is repeated. In some embodiments, if theanatomy does not need to be recalibrated, then the process beginning atstep 450 is repeated.

In some embodiments, at any time during the procedure, certain faultconditions may cause the computer 100 to interrupt the program andrespond accordingly. For instance, in some embodiments, if the signalfrom the RF transmitters 120 cannot be read, then the computer 100 maybe programmed to stop the movement of the robot 15 or remove thesurgical instrument 35 from the patient 18. Another example of a faultcondition is if the robot 15 encounters a resistance above apreprogrammed tolerance level.

Referring now to FIG. 14, a flow chart diagram 1400 for a conventionalflexible catheter or wire insertion procedure according to an embodimentof the invention is shown. Catheters are used in a variety of medicalprocedures to deliver medicaments to a specific site in a patient'sbody. Often, delivery to a specific location is needed so a targeteddiseased area can then be treated. Sometimes instead of inserting thecatheter directly, a flexible wire is first inserted, over which theflexible catheter can be slid.

In some embodiments, the distance between each of the calibratingtransmitters 120 relative to each other is measured prior to calibrationstep 500. In some embodiments, each calibrating transmitter 120transmits RF signals on a different frequency so the positioning sensors12, 110 can determine which transmitter 120 emitted a particular RFsignal. In some embodiments, the signal from each of these transmitters120 is received by positioning sensors 12, 110. Since the distancebetween each of the calibrating transmitters 120 is known, and thesensors can identify the signals from each of the calibratingtransmitters 120 based on the known frequency, in some embodiments,using time of flight calculation, the positioning sensors 12, 110 areable to calculate the spatial distance of each of the positioningsensors 12, 110 relative to each other. The system is now calibrated. Asa result, in some embodiments, the positioning sensors 12, 110 can nowdetermine the spatial position of any new RF transmitter 120 introducedinto the room 10 relative to the positioning sensors 12, 110.

In some embodiments, at step 510, reference needles that contain the RFtransmitters 120 are inserted into the body. The purpose of theseneedles is to track movement of key regions of soft tissue that willdeform during the procedure or with movement of the patient 18.

In some embodiments, at step 520, a 3D anatomical image scan (such as aCT scan) is taken of the anatomical target. Any 3D anatomical image scanmay be used with the surgical robot 15 and is within the scope of thepresent invention. In some embodiments, the anatomical image capturearea includes the tips of the reference needles so that theirtransmitters' 120 positions can be determined relative to the anatomy.

In some embodiments, at step 530, the RF signals from the catheter tipand reference needles are read.

In some embodiments, at step 540, the position of the catheter tip iscalculated. Because the position of the catheter tip relative to thereference needles and the positions of the reference needles relative tothe anatomy are known, the computer 100 can calculate the position ofthe catheter tip relative to the anatomy.

In some embodiments, at step 550, the superimposed catheter tip and theshaft representation is displayed on the anatomical image taken in step520.

In some embodiments, at step 560, the computer 100 determines whetherthe catheter tip is advancing toward the anatomical target. If it is notmoving to the anatomical target, then step 563 is reached. If it iscorrectly moving, then step 570 is reached.

In some embodiments, at step 563, the robot 15 arm is adjusted to guidethe catheter tip in the desired direction. If the anatomy needs to becalibrated, then in some embodiments, the process beginning at step 520is repeated. If the anatomy does not need to be recalibrated, then theprocess beginning at step 540 is repeated.

In some embodiments, at step 570, the computer 100 determines whetherthe procedure has been completed. If the procedure has not beencompleted, then the process beginning at step 540 is repeated.

In some embodiments, at any time during the procedure, certain faultconditions may cause the computer 100 to interrupt the program andrespond accordingly. For instance, in some embodiments, if the signalfrom the RF transmitter's 120 cannot be read, then the computer 100 maybe programmed to stop the movement of the robot 15 or remove theflexible catheter from the patient 18. Another example of a faultcondition is if the robot 15 encounters a resistance above apreprogrammed tolerance level. A further example of a fault condition isif the RF transmitter's 120 on the anatomical target indicate thelocation of the anatomical target shift so that actual and calculatedpositions of the anatomy no longer match. In some embodiments, oneindicator that the anatomical target location has shifted relative tothe transmitter's 120 is if the computer 100 calculates that thesurgical instrument 35 appears to be inside bone when no drilling orpenetration is actually occurring.

In some embodiments, the proper response to each condition may beprogrammed into the system, or a specific response may beuser-initiated. For example, in some embodiments, the computer 100 maydetermine that in response to an anatomy shift, the anatomy would haveto be recalibrated, and the process beginning at step 520 should berepeated. Alternatively, in some embodiments, a fault condition mayrequire the flowchart to repeat from step 500. In other embodiments, theuser may decide that recalibration from step 520 is desired, andinitiate that step himself.

Referring now to FIGS. 15A & 15B, screenshots of software for use withthe described system is provided in accordance with some embodiments ofthe invention. The software provides the method to select the targetarea of surgery, plan the surgical path, check the planned trajectory ofthe surgical path, synchronize the medical images to the positioningsystem and precisely control the positioning system during surgery. Thesurgical positioning system and navigation software includes an opticalguidance system or RF Local Positioning System (RF-LPS), which are incommunication with the positioning system.

FIG. 15A shows a screen shot 600 of the selection step for a user usinga software program as described herein in accordance with someembodiments of the invention. Screen shot 600 includes windows 615, 625,and 635, which show a 3D anatomical image of surgical target 630 ondifferent planes. In this step, the user selects the appropriate 3Dimage corresponding to anatomical location of where the procedure willoccur. In some embodiments, the user uses a graphic control to changethe perspective of the image in order to more easily view the image fromdifferent angles. In some embodiments, the user can view the surgicaltarget 630 within separate coordinated views for each of the x-axis,y-axis and z-axis coordinates for each anatomical location in thedatabase in each window 615, 625 and 635, respectively.

In some embodiments, after selecting the desired 3D image of thesurgical target 630, the user will plan the appropriate trajectory onthe selected image. In some embodiments, an input control is used withthe software in order to plan the trajectory of the surgical instrument35. In one embodiment of the invention, the input control is in theshape of a biopsy needle 8110 for which the user can plan a trajectory.

FIG. 15B shows a screen shot 650 during the medical procedure inaccordance with some embodiments of the invention. In some embodiments,the user can still view the anatomical target 630 in different x-axis,y-axis and z-axis coordinate views on windows 615, 625, and 635. Asshown in screen shot 650, the user can see the planned trajectory line670 in multiple windows 615 and 625. The actual trajectory and locationof the surgical instrument 35 is superimposed on the image (shown asline segment 660). In some embodiments, the actual trajectory andlocation of the surgical instrument 35 is dynamically updated anddisplayed, and is shown as a line segment 660. In some otherembodiments, the actual trajectory and location of the surgicalinstrument 35 could be shown as a trapezoid or a solid central linesurrounded by a blurred or semi-transparent fringe to represent theregion of uncertainty. In some embodiments (under perfect conditionswith no bending of the surgical instrument as it enters tissues) thetracking system 3417 and robot 15 encoders calculate that the surgicalinstrument 35 should be located at the solid line or center of thetrapezoid. In some embodiments, due to bending of the instrument 35 thatmight occur if tissues of different densities are crossed, there mightbe bending, with the amount of reasonably expected bending displayed asthe edges of the trapezoid or fringe. In some embodiments, the size ofthis edge could be estimated knowing the stiffness and tolerance of thesurgical instrument 35 within the guide tube 50, and by usingexperimental data collected for the same instrument 35 under previouscontrolled conditions. In some embodiments, displaying this region ofuncertainty helps prevent the user from expecting the system to delivera tool to a target trajectory with a physically impossible level ofprecision.

As described earlier, in some embodiments, the surgical robot 15 can beused with alternate guidance systems other than an LPS. In someembodiments, the surgical robot system 1 can comprise a targetingfixture 690 for use with a guidance system. In some embodiments, onetargeting fixture 690 comprises a calibration frame 700, as shown inFIGS. 20A-20E. A calibration frame 700 can be used in connection withmany invasive procedures; for example, it can be used in thoracolumbarpedicle screw insertion in order to help achieve a more accuratetrajectory position. In some embodiments, the use of the calibrationframe 700 can simplify the calibration procedure. In some embodiments ofthe invention, the calibration frame 700 can be temporarily affixed tothe skin of a patient 18 surrounding a selected site for a medicalprocedure, and then the medical procedure can be performed through awindow defined by the calibration frame.

As shown in FIGS. 20A and 20B, in some embodiments of the invention, thecalibration frame 700 can comprise a combination of radio-opaque markers730 and infrared, or “active,” markers 720. In some embodiments, theradio-opaque markers 730 can be located within the CT scan region 710,and the active markers 720 can be located outside of the CT scan region710. In some embodiments, a surgical field 17 (i.e., the area where theinvasive procedure will occur) can be located within the perimetercreated by radio-opaque markers 730. In some embodiments, the actualdistances of the radio-opaque 730 and active markers 720 relative toeach other can be measured from a high-precision laser scan of thecalibration frame. Additionally or alternatively, in some embodiments,the actual relative distances can be measured by actively measuring thepositions of active markers 720 while nearly simultaneously orsimultaneously pointing with a pointing device, such as a conventionaldigitizing probe, to one or more locations on the surface of theradio-opaque markers 730. In certain embodiments, digitizing probes cancomprise active markers 720 embedded in a rigid body 690 and a tipextending from the rigid body.

In some embodiments, through factory calibration or other calibrationmethod(s), such as pivoting calibration, the location of the probe tiprelative to the rigid body of the probe can be established. In someembodiments, it can then be possible to calculate the location of theprobe's tip from the probe's active markers 720. In some embodiments,for a probe with a concave tip that is calibrated as previouslydescribed, the point in space returned during operation of the probe canrepresent a point distal to the tip of the probe at the center of thetip's concavity. Therefore, in some embodiments, when a probe(configured with a concave tip and calibrated to marker 730 of the sameor nearly the same diameter as the targeting fixture's radio-opaquemarker 730) is touched to the radio-opaque marker 730, the probe canregister the center of the sphere. In some embodiments, active markers720 can also be placed on the robot in order to monitor a position ofthe robot 15 and calibration frame 700 simultaneously or nearlysimultaneously.

In some embodiments, the calibration frame 700 is mounted on thepatient's skin before surgery/biopsy, and will stay mounted during theentire procedure. Surgery/biopsy takes place through the center of theframe 700.

In some embodiments, when the region of the plate with the radio-opaquemarkers 730 is scanned intra-operatively or prior to surgery (forexample, using a CT scanner), the CT scan contains both the medicalimages of the patient's bony anatomy, and spherical representations ofthe radio-opaque markers 730. In some embodiments, software is used todetermine the locations of the centers of the markers 730 relative tothe trajectories defined by the surgeon on the medical images. Becausethe pixel spacing of the CT scan can be conveyed within encoded headersin DICOM images, or can be otherwise available to a tracking software(for example, the robotic guidance software 3406), it can, in someembodiments, be possible to register locations of the centers of themarkers 730 in Cartesian coordinates (in millimeters, for example, orother length units). In some embodiments, it can be possible to registerthe Cartesian coordinates of the tip and tail of each trajectory in thesame length units.

In some embodiments, because the system knows the positions of thetrajectories relative to the radio-opaque markers 730, the positions ofthe radio-opaque markers 730 relative to the active markers 720, and thepositions of the active markers 720 on the calibration frame 700relative to the active markers on the robot 15 (not shown), the systemhas all information necessary to position the robot's end-effectuator 30relative to the defined trajectories.

In some other embodiments of the invention, the calibration frame 700can comprise at least three radio-opaque markers 730 embedded in theperiphery of the calibration frame 700. In some embodiments, the atleast three radio-opaque markers 730 can be positioned asymmetricallyabout the periphery of the calibration frame 700 such that the software,as described herein, can sort the at least three radio-opaque markers730 based only on the geometric coordinates of each marker 730. In someembodiments, the calibration frame 700 can comprise at least one bank ofactive markers 720. In some embodiments, each bank of the at least onebank can comprise at least three active markers 720. In someembodiments, the at least one bank of active markers 720 can comprisefour banks of active markers 720. In yet another aspect, the calibrationframe 700 can comprise a plurality of leveling posts 77 coupled torespective corner regions of the calibration frame 700. In someembodiments, the corner regions of the calibration frame 700 can includeleveling posts 77 that can comprise radiolucent materials. In someembodiments, the plurality of leveling posts 77 can be configured topromote uniform, rigid contact between the calibration frame 700 and theskin of the patient 18. In some embodiments, a surgical-grade adhesivefilm, such as, for example and without limitation, Ioban™ from 3M™, canbe used to temporarily adhere the calibration frame 700 to the skin ofthe patient 18. 3M™ and Ioban™ are registered trademarks of 3M Company.In some further embodiments, the calibration frame 700 can comprise aplurality of upright posts 75 that are angled away from the frame 700(see FIG. 20B). In some embodiments, the plurality of active markers 720can be mounted on the plurality of upright posts 75.

As shown in FIG. 20B, in some embodiments, there are four radio-opaquemarkers 730 (non-metallic BBs from an air gun) embedded in the peripheryof the frame, labeled OP1, OP2, OP3, OP4. In some embodiments, onlythree markers 730 are needed for determining the orientation of a rigidbody in space (the 4th marker is there for added accuracy).

In some embodiments, the radio-opaque markers 730 are placed in anasymmetrical configuration (notice how OP1 and OP2 are separated fromeach other by more distance than OP3 and OP4, and OP1 and OP4 arealigned with each other across the gap, however OP3 is positioned moretoward the center than OP2). The reason for this arrangement is so thata computer algorithm can automatically sort the markers to determinewhich is which if only given the raw coordinates of the four markers andnot their identification.

In some embodiments, there are four banks of active markers 720 (threemarkers 720 per bank). Only one bank of three markers 720 is needed(redundancy is for added accuracy and so that the system will still workif the surgeon, tools, or robot are blocking some of the markers.

In some embodiments, despite the horizontal orientation of the patient18, the angulation of the upright posts can permit the active markers720 to face toward the cameras (for example cameras 8200 shown in FIG.81). or detection devices of the tracking system (for example, thetracking system 3417). In some embodiments, the upright posts can beangled away from the calibration frame by about 10°.

In some applications, to establish the spatial relationship between theactive 720 and radio-opaque markers 730, a conventional digitizingprobe, such as a 6-marker probe, embedded with active markers 720 in aknown relationship to the probe's tip (see for example FIG. 20C) can beused to point to each of the radio-opaque markers 730. In someembodiments, the probe can point to locations on two opposite surfacesof the spherical radio-opaque markers 730 while recording the positionof the probe tip and the active markers 720 on the frame 700simultaneously. Then, the average position of the two surfacecoordinates can be taken, corresponding to the center of the sphere. Animage of the robot 15 used with this targeting fixture 690 is shown inFIG. 20D. For placement of conventional surgical screws, a biopsy,injection, or other procedures, in some embodiments, the robot 15 canwork through the window formed by the frame 700. During a surgicalprocedure, in some embodiments, the working portal is kept on theinterior of the frame 700 and the markers 720 on the exterior of theframe 700 can improve accuracy over a system where fiducials are mountedaway from the area where surgery is being performed. Without wishing tobe bound by theory, simulation, and/or modeling, it is believed that areason for improved accuracy is that optimal accuracy of trackingmarkers 720 can be achieved if tracking markers 720 are placed aroundthe perimeter of the frame 700 being tracked.

Further embodiments of the invention are shown in FIG. 20E illustratinga calibration frame 700. This fixture 690 is simplified to make it lessobstructive to the surgeon. In some embodiments, the calibration frame700 can comprise four active markers 720 having a lower profile than theactive markers 720 described above and depicted in FIGS. 20A-20D. Forexample, the calibration frame 700 can comprise a plurality of uprightposts 75 that are angled away from the calibration frame by about 10°.In some embodiments, the active markers 720 are mounted on the posts 75that are angled back by 10°, and this angulation keeps the markers 720facing toward the cameras despite the patient being horizontal.

Moreover, in some embodiments, the front markers 720 can have lesschance of obscuring the rear markers 720. For example, posts 75 that arefarthest away from the camera or farthest from a detection device of thetracking system 3417 can be taller and spaced farther laterally than theposts 75 closest to the camera.

In some further embodiments of the invention, the calibration frame 700can comprise markers 730 that are both radio-opaque for detection by amedical imaging scanner, and visible by the cameras or otherwisedetectable by the real-time tracking system 3417. In some embodiments,the relationship between radio-opaque 730 and active markers (730, 720)does not need to be measured or established because they are one in thesame. Therefore, in some embodiments, as soon as the position isdetermined from the CT scan (or other imaging scan), the spatialrelationship between the robot 15 and anatomy of the patient 18 can bedefined.

In other embodiments, the targeting fixture 690 can comprise a flexibleroll configuration. In some embodiments, the targeting fixture 690 cancomprise three or more radio-opaque markers 730 that define a rigidouter frame and nine or more active markers 720 embedded in a flexibleroll of material (for example, the flexible roll 705 in FIG. 21A). Asdescribed earlier, radio-opaque markers 730 are visible on CT scansand/or other medical diagnostic images, such as MRI, or reconstructionsfrom O-arm or Iso-C scans, and their centroids can be determined fromthe 3D image. Active markers 720 include tracked markers 720 that have3D coordinates that are detectable in real-time using cameras or othermeans. Some embodiments can utilize active marker systems based onreflective optical systems such as Motion Analysis Inc., or PeakPerformance. Other suitable technologies include infrared-emittingmarker systems such as Optotrak, electromagnetic systems such asMedtronic's Axiem®, or Flock of Birds®, or a local positioning system(“LPS”) described by Smith et al. in U.S. Patent Publication No.2007/0238985.

Flock Of Birds® is a registered trademark of Ascension TechnologyCorporation.

Axiem is a trademark of Medtronic, Inc., and its affiliated companies.

Medtronic® is a registered trademark used for Surgical and MedicalApparatus, Appliances and Instruments.

In some embodiments of the invention, at least a portion of the flexibleroll 705 can comprise self-adhering film, such as, for example andwithout limitation, 3M™Ioban™ adhesive film (iodine-impregnatedtransparent surgical drape) similar to routinely used operating roomproduct model 6651 EZ (3M, St. Paul, Minn.). Ioban™ is a trademark of 3Mcompany.

In some embodiments, within the flexible roll 705, the radio-opaque andactive markers (730, 720) can be rigidly coupled to each other, witheach radio-opaque marker 730 coupled to three or more active markers720. Alternatively, in some embodiments, the markers can simultaneouslyserve as radio-opaque and active markers (for example, an active marker720 whose position can be detected from cameras or other sensors), andthe position determined from the 3D medical image can substantiallyexactly correspond to the center of the marker 720. In some embodiments,as few as three such markers 720 could be embedded in the flexible roll705 and still permit determination of the spatial relationship betweenthe robot 15 and the anatomy of the patient 18. If radio-opaque markers730 and active markers 720 are not one in the same, in some embodimentsthe at least three active markers 720 must be rigidly connected to eachradio-opaque marker 730 because three separate non-collinear points areneeded to unambiguously define the relative positions of points on arigid body. That is, if only one or 2 active markers 720 are viewed,there is more than one possible calculated position where a rigidlycoupled radio-opaque marker could be.

In some embodiments of the invention, other considerations can be usedto permit the use of two active markers 720 per radio-opaque marker 730.For example, in some embodiments, if two active markers 720 and oneradio-opaque marker 730 are intentionally positioned collinearly, withthe radio-opaque marker 730 exactly at the midpoint between the twoactive markers 720, the location of the radio-opaque marker 730 can bedetermined as the mean location of the two active markers 720.Alternatively, in some embodiments, if the two active markers 720 andthe radio-opaque marker 730 are intentionally positioned collinearly butwith the radio-opaque marker 730 closer to one active marker 720 thanthe other (see for example FIG. 21B), then in some embodiments, theradio-opaque marker 730 must be at one of two possible positions alongthe line in space formed by the two active markers 720 (see FIG. 21B).In this case, in some embodiments, if the flexible roll 705 isconfigured so that each pair of active markers 720 is oriented (when inits final position) with one marker 720 more toward the center of theflexible roll 705, then it can be determined from the orientations ofall markers or certain combinations of markers from different regionswhich of the two possible positions within each region is the correctposition for the radio-opaque marker 730 (see FIG. 21C showing flexibleroll 705 showed rolled on a torso and shown unrolled on a torso withmarkers 720, 730 in place). As shown, the radio-opaque markers 730 canbe positioned toward the inside of the frame 705), with marker groupsnearer to the top of the figure having the radio-opaque marker 730positioned below the active markers 720 and marker groups near thebottom of the figure having the radio-opaque marker positioned above theactive markers 720.

In some embodiments, the flexible roll 705 can be positioned across thepatient's back or other area, and adhered to the skin of the patient 18as it is unrolled. In some embodiments, knowing the spatial relationshipbetween each triad of active markers 720 and the rigidly coupledradio-opaque marker 730, it is possible to establish the relationshipbetween the robot 15 (position established by its own active markers720) and the anatomy (visualized together with radio-opaque markers 730on MM, CT, or other 3D scan). In some embodiments, the flexible roll 705can be completely disposable. Alternatively, in some other embodiments,the flexible roll 705 can comprise reusable marker groups integratedwith a disposable roll with medical grade adhesive on each side toadhere to the patient 18 and the marker groups 720, 730. In some furtherembodiments, the flexible roll 705 can comprise a drape incorporatedinto the flexible roll 705 for covering the patient 18, with the drapeconfigured to fold outwardly from the roll 705.

In some embodiments, after the roll 705 has been unrolled, the roll 705can have a desired stiffness such that the roll 705 does notsubstantially change its position relative to the bony anatomy of thepatient 18. In some embodiments of the invention, a conventionalradiolucent wire can be embedded in the perimeter of the frame 700. Insome embodiments, it a chain of plastic beads, such as the commerciallyavailable tripods shown in FIG. 21D, or a commercially available “snakelight” type fixture, can be employed to provide desired stiffness to theunrolled fixture such that it maintains its position after unrollingoccurs. For example, in some embodiments, the beads of the chain ofplastic beads as shown can be affixed to each other with a high frictionso that they hold their position once shifted. Further, in someembodiments, chains of beads can be incorporated into, and define, aperimeter of the frame 700. In some embodiments, this type of framecould be loaded with conventional chemicals that mix at the time ofapplication. For example, in some embodiments, components of aconventional two-part epoxy could be held in separate fragile baggieswithin the frame that pop open when the user first starts to manipulatethe beads. In some embodiments, the user would attach the frame to thepatient 18, and mold it to the contours of the patient's body. After ashort period of time, the frame 700 would solidify to form a very rigidframe, locking the beads in their current orientation.

In some embodiments of the invention, the targeting fixture 690 can bean adherable fixture, configured for temporary attachment to the skin ofa patient 18. For example, in some embodiments, the targeting fixture690 can be temporarily adhered to the patient 18 during imaging,removed, and then subsequently reattached during a follow-up medicalprocedure, such as a surgery. In some embodiments, the targeting fixture690 can be applied to the skull of a patient 18 for use in placement ofelectrodes for deep brain stimulation. In some embodiments, this methodcan use a single fixture 690, or two related fixtures. In this instance,the two related fixtures can share the same surface shape. However, onefixture 690 can be temporarily attached at the time of medical imagescanning, and can include radio-opaque markers 730 (but not activemarkers 720), and the second fixture 690 can be attached at the time ofsurgery, and can include active markers 720 (but not radio-opaquemarkers 730).

In some embodiments, the first fixture (for scanning) can comprise aframe 690 with three or more embedded radio-opaque markers 730, and twoor more openings 740 for application of markings (the markings shown as750 in FIG. 22B). In some embodiments, the device 690 can be adhered tothe scalp of a patient 18, and the openings 740 can be used to paintmarks on the scalp with, for example, henna or dye (shown as “+” marks750 in FIG. 22B). With this fixture 690 in place, in some embodiments,the patient 18 can receive a 3D scan (for example an MRI or CT scan) inwhich the radio-opaque markers 730 are captured. As illustrated by FIG.22B, in some embodiments, the fixture 690 can then be removed, leavingthe dye marks 750 on the scalp. In some embodiments, on a later date(before the dye marks 750 wear off), the patient 18 can return, and thesurgeon or technician can attach the 2nd fixture (for surgery)containing active markers 720 for intraoperative tracking (see FIG.22C-22D). In some embodiments, the fixture 690 shown in FIG. 22C can bemounted to the scalp, spatially positioned and oriented in the sameposition as the previously adhered first fixture (shown in FIG. 22A) byensuring that the previously placed dye marks 750 line up with holes inthe second fixture 690 (see the alignment arrows depicted in FIG. 22C).Optionally, in some embodiments, the fixture 690 can have a transparentframe for good visualization. The above-described method assumes thatthe locations of the marks 750 do not change over the period of timebetween the scan and the return of the patient 18 for surgery. Since therelative positions between the radio-opaque markers 730 from thetemporary (first) fixture 690 (which appear in the scan) and the activemarkers 720 on the second applied fixture 690 are known through acalibration and/or by careful manufacturing of the fixtures 690, thecoordinate system of the anatomy and the coordinate system of the activemarkers 720 can be synchronized so that the robot 15 can target anyplanned trajectory on the 3D image as described further herein. Further,in some embodiments, this method can enable image guidance with only onepre-op scan and without requiring the patient 18 to go home after apre-op scan. This circumvents the need for a patient 18 to take care ofwounds from targeting screws that are invasively drilled into the skullof the patient 18.

In some embodiments of the invention, the targeting fixture 690 cancomprise a conventional clamping mechanism for securely attaching thetargeting fixture 690 to the patient 18. For example, in someembodiments, the targeting fixture 690 can be configured to clamp to thespinous process 6301 of a patient 18 after the surgeon has surgicallyexposed the spinous process. FIG. 23 shows a dynamic tracking device2300 mounted to the spinous process 2310 in the lumbar spine of apatient 18 in accordance with some embodiments of the invention. Thistargeting fixture is used with Medtronic's StealthStation. This figureis reprinted from Bartolomei J, Henn J S, Lemole G M Jr., Lynch J,Dickman C A, Sonntag V K H, Application of frameless stereotaxy tospinal surgery, Barrow Quarterly 17(1), 35-43 (2001).

StealthStation® is a trademark of Medtronic, Inc., and its affiliatedcompanies.

In some embodiments, during use of a targeting fixture 690 having aconventional clamping mechanism with image guidance, the relationshipbetween the markers 720, 730 and the bony anatomy of the patient 18 canbe established using a registration process wherein known landmarks aretouched with a digitizing probe at the same time that the markers on thetracker are visible. In some embodiments of the invention, the probeitself can have a shaft protruding from a group of markers 720, 730,thereby permitting the tracking system 3417 to calculate the coordinatesof the probe tip relative to the markers 720, 730.

In some embodiments, the clamping mechanism of the targeting fixture 690can be configured for clamping to the spinous process 2310, or can beconfigured for anchoring to bone of the patient 18 such that the fixture690 is substantially stationary and not easily moved. In some furtherembodiments, the targeting fixture 690 can comprise at least threeactive markers 720 and distinct radio-opaque markers 730 that aredetected on the CT or other 3D image, preferably near the clamp (to beclose to bone). In some alternative embodiments, the active markers 720themselves must be configured to be visualized accurately on CT or other3D image. In certain embodiments, the portion of the fixture 690containing a radio-opaque marker 730 can be made to be detachable toenable removal from the fixture after the 3D image is obtained. In somefurther embodiments, a combination of radio-opaque 730 and activemarkers 720 can allow tracking with the robot 15 in the same way that ispossible with the frame-type targeting fixtures 690 described above.

In some embodiments, one aspect of the software and/or firmwaredisclosed herein is a unique process for locating the center of theabove-described markers 730 that takes advantage of the fact that a CTscan can comprise slices, typically spaced 1.5 mm or more apart in the zdirection, and sampled with about 0.3 mm resolution in the x-axis andy-axis directions. In some embodiments, since the diameter of theradio-opaque markers 730 is several times larger than this slicespacing, different z slices of the sphere will appear as circles ofdifferent diameters on each successive x-y planar slice. In someembodiments, since the diameter of the sphere is defined beforehand, thenecessary z position of the center of the sphere relative to the slicescan be calculated to provide the given set of circles of variousdiameters. Stated similarly, in some embodiments, a z slicesubstantially exactly through the center of the sphere can yield acircle with a radius R that is substantially the same as that of thesphere. In some embodiments, a z slice through a point at the top orbottom of the sphere can yield a circle with a radius R approximatingzero. In some other embodiments, a z slice through a z-axis coordinateZ1 between the center and top or bottom of the sphere can yield a circlewith a radius R1=R cos(arcsin(Z1/R)).

In some embodiments of the invention, the observed radii of circles on zslices of known inter-slice spacing can be analyzed using the equationdefined by R1=R cos(arcsin(Z1/R)). This provides a unique mathematicalsolution permitting the determination of the distance of each slice awayfrom the center of the sphere. In cases in which a sphere has a diametersmall enough that only a few slices through the sphere appear on amedical image, this process can provide a more precise the center of asphere.

Some embodiments of the use of the calibration frame 700 are describedto further clarify the methods of use. For example, some embodimentsinclude the steps of a conventional closed screw or conventional needle(for example, a biopsy needle 8110) insertion procedure utilizing acalibration frame 700 as follows. In some embodiments, a calibrationframe 700 is attached to the patient's 18 skin, substantially within theregion at which surgery/biopsy is to take place. In some embodiments,the patient 18 receives a CT scan either supine or prone, whicheverpositioning orients the calibration frame 700 upward. In someembodiments, the surgeon subsequently manipulates three planar views ofthe patient's 18 CT images with rotations and translations. In someembodiments, the surgeon then draws trajectories on the images thatdefine the desired position, and strike angle of the end-effectuator 30.In some embodiments, automatic calibration can be performed in order toobtain the centers of radio-opaque makers 730 of the calibration frame700, and to utilize the stored relationship between the active markers720 and radio-opaque markers 730. This procedure permits the robot 15 tomove in the coordinate system of the anatomy and/or drawn trajectories.

In some embodiments, the robot 15 then will move to the desiredposition. In some embodiments, if forceful resistance beyond a pre-settolerance is exceeded, the robot 15 will halt. In some furtherembodiments, the robot 15 can hold the guide tube 50 at the desiredposition and strike angle to allow the surgeon to insert a conventionalscrew or needle (for example, needle 7405, 7410 or biopsy needle 8110).In some embodiments, if tissues move in response to applied force or dueto breathing, the movement will be tracked by optical markers 720, andthe robot's position will automatically be adjusted.

As a further illustration of a procedure using an alternate guidancesystem, in some embodiments, the steps of an open screw insertionprocedure utilizing an optical guidance system is described. In someembodiments, after surgical exposure, a targeting fixture 690 comprisinga small tree of optical markers, for example, can be attached to a bonyprominence in the area of interest. In some embodiments, conventionalcalibration procedures for image guidance can be utilized to establishthe anatomy relative to the optical tracking system 3417 and medicalimages. For another example, the targeting fixture 690 can containrigidly mounted, substantially permanent or detachable radio-opaquemarkers 730 that can be imaged with a CT scan. In some embodiments, thecalibration procedures consistent with those stated for the calibrationframe 700 can be utilized to establish the anatomy relative to the robot15 and the medical image.

In some embodiments, the surgeon manipulates three planar views of thepatient's CT images with rotations and translations. In someembodiments, the surgeon then draws trajectories on the images thatdefine the desired position and strike angle of the end-effectuator 30.In some embodiments, the robot 15 moves to the desired position. In someembodiments, if forceful resistance beyond a pre-set tolerance isexceeded, the robot 15 will halt. In some embodiments, the robot 15holds the guide tube 50 at the desired position and strike angle toallow the surgeon to insert a conventional screw. In some embodiments,if tissues move in response to applied force or due to breathing, themovement will be tracked by optical markers 720, and the robot'sposition will automatically be adjusted.

FIG. 36 illustrates an example embodiment 3600 of surgical robot system1 that utilizes a surveillance marker 710 in accordance with one or moreaspects of the invention. As illustrated, the example embodiment 3600comprises a 4-marker tracker array 3610 attached to the patient 18 andhaving a surveillance marker, and a 4-marker tracker array 3620 on therobot 15. In some embodiments, during usage, it may possible that atracker array, or tracker (3610 in FIG. 36), on a patient 18inadvertently shifts. For example, a conventional clamp positioned on apatient's 18 spinous process 2310 where the tracker 3610 is attached canbe bumped by the surgeon's arm and move (i.e., bend or translate) to anew position relative to the spinous process 2310. Alternatively, atracker 3610 that is mounted to the skin of the patient 18 can movegradually with the skin, as the skin settles or stretches over time. Inthis instance, the accuracy of the robot 15 movement can be lost becausethe tracker 3610 can reference bony anatomy from a medical image that nolonger is in the same position relative to the tracker as it had beenduring the medical image scan. To overcome such problems, someembodiments of the invention provide a surveillance marker 710 asillustrated in FIG. 36. As shown, in some embodiments, the surveillancemarker 710 can be embodied or can comprise one or more markers 710rigidly affixed to a patient 18 in a location different than thelocation in which a primary tracker array 3610 is affixed; for example,a different spinous process 2310, on the skin, or on a small postdrilled into the ilium. Accordingly, in some embodiments, thesurveillance marker 710 can be located on the same rigid body as theprimary tracker array 3610 but at a different location on the rigidbody.

In one embodiment, in response to placement of the surveillance marker710, execution of a control software application (e.g., robotic guidancesoftware 3406) can permit an agent (e.g., a surgeon, a nurse, adiagnostician) to select “set surveillance marker”. At this time, thevector (3D) distances between the surveillance marker 710, and each ofthe markers 3611, 3612, 3613, and 3614 on the primary tracker array 3610can be acquired and retained in computer 100 memory (such as a memory ofa computing device 3401 executing the control software application). Inan embodiment in which a 4-marker tracker array 3610 is utilized (FIG.36), four distances 3611 a, 3612 a, 3613 a, and 3614 a can be acquiredand retained, representing the distances between the surveillance marker710 and markers 3611, 3612, 3613, and 3614. In such embodiment, at eachframe of real-time data during a procedure, the surgical robot system 1disclosed herein can calculate updated distances between each of themarkers 3611, 3612, 3613, and 3614 on the primary tracker array 3610 andthe surveillance marker 710. The system 1 can then compare the updateddistances or a metric thereof (for example, the sum of the magnitude ofeach distance) to the available values (for example, values retained inthe computer 100 memory). In some embodiments, in view that thesurveillance marker 710 and tracker array 3610 can be on the same rigidbody, the updated distances and/or the metric thereof (such as theirsum) can remain substantially fixed unless one or more of the trackerarray 3610 or the surveillance marker 710 shifts. In some embodiments,in response to a shift of the tracker array 3610 or the surveillancemarker 710, or both, a notification can be issued to alert an agent of aloss in movement accuracy. In some embodiments, if the surveillancemarker 710 offset exceeds a pre-set amount, operation of the surgicalrobot system 1 can be halted. In some embodiments, in response to a userintentionally shifting the tracker array 3610 or the surveillance marker710 to a new position, execution of the control software application canpermit overwriting a set of one or more stored distances with new valuesfor comparison to subsequent frames.

In some embodiments, as illustrated in FIG. 36, embodiment 3600 ofsurgical robot system 1 utilizes a surveillance marker 710, a 4-markertracker array 3610 attached to the patient 18, and a 4-marker trackerarray 3620 on the robot 15. It should be appreciated that in someembodiments, the 4-marker tracker array 3620 on the robot 15 canexperience an unintentional shift in a manner similar to that for the4-marker array tracker 3610 on the patient 18. Consequently, in certainembodiments, a surveillance marker (not shown) can be attached to adifferent position on the robot 15 arm than the robot's 4-marker trackerarray 3620 to control, at least in part, such unintentional shift. Insome embodiments, a surveillance marker on the robot 15 may providelesser efficiencies than a surveillance marker 710 on the patient 18 inview that the robot 15 arm can be manufactured with negligible orminimal likelihood of the robot's tracker array 3620 or surveillancemarker (not shown) shifting. In addition or in the alternative, otherembodiments can include means for registering whether the tracker 3620has shifted can be contemplated for the robot's tracker array 3620. Forinstance, in some embodiments, the means for registering may not includea surveillance marker, but may comprise the extant robot 15 trackingsystem 3417 and one or more of the available conventional encoders. Insome embodiments, the system 1 and encoder(s) can compare movementregistered from the tracker 3620 to movement registered from counts ofencoders (not shown) on each robot 15 axis. For example, in someembodiments where the robot's tracker array 3620 is mounted on thehousing 27 that rotates with the roll 62 axis (which can be farther awayfrom the base 25 than the z-axis 70, x-axis 66, y-axis 68, and roll 62axis) then changes in z-axis 70, x-axis 66, y-axis 68, and roll 62 axisencoder counts can provide highly predictable changes in the position ofthe robot's tracker array 3620 in the coordinate systems of the trackingsystem 3417 and robot 15. In some embodiments, the predicted movementbased on encoder counts and tracked 3D position of the tracker array3620 after application of known counts can be compared and, if thevalues differ substantially (or values are above a predeterminedthreshold), the agent can be alerted to the existence of that anoperational issue or malfunction. The operational issue can originatefrom one or more of a malfunction in the registration of counts (i.e.,electromechanical problem), malfunction in registration of the tracker'smarkers 3621, 3622, 3623, 3624 (for example, outside of trackingsystem's optimum volume), or shift in the position of the tracker 3620on the robot's 15 surface during the move.

It should be appreciated that other techniques (for example, methods,systems, and combinations thereof, or the like) can be implemented inorder to respond to operational issues that may prevent tracking of themovement of a robot 15 in the surgical robot system 1. In oneembodiment, marker reconstruction can be implemented for steadiertracking. In some embodiments, marker reconstruction can maintain therobot end-effectuator 30 steady even if an agent partially blocksmarkers during operation of the disclosed surgical robot system 1.

As described herein, in some embodiments, at least some features oftracking movement of the robot's end-effectuator 30 can comprisetracking a virtual point on a rigid body utilizing an array of one ormore markers 720, such tracking comprising one or more sequences oftranslations and rotations. As an illustration, an example methodologyfor tracking a visual point on a rigid body using an array of threeattached markers is described in greater detail herein, such methodologycan be utilized to implement marker reconstruction technique inaccordance with one or more aspects of the invention. FIG. 37, forexample, illustrates an example of a methodology for tracking a visualpoint 4010 on a rigid body using an array of three attached markers4001, 4002, 4003. In some embodiments, the method includes contemplatinga reference data-frame. In some embodiments, the reference data-framecan be associated with a set of reproducible conditions for the relativepositions of the markers 4001, 4002, 4003, but not necessarily definedlocations.

FIG. 38 illustrates a procedure for monitoring the location of a pointof interest 4010 relative to three markers 4001, 4001, 4003 based onimages received form the methodology illustrated in FIG. 37 inaccordance with some embodiments of the invention. As shown, the methodincludes translation and rotating the markers 4001, 4002, 4003 and pointof interest 4010 with the conditions as shown. In some embodiments, themethod can include saving the x-axis, y-axis, and z-axis coordinates ofthe point of interest 4010 in this reference frame for future use. Insome embodiments, for each subsequent data-frame the method can includethe steps of; 1). transform the markers 4001, 4002, 4003 using theconditions defined for the reference frame (keeping track of therotations and translations), 2). add the point of interest 4010 (whichwas saved after establishing the reference frame) and 3). transform thepoint of interest 4010 back to the current location of the markers 4001,4002, 4003 using inverses of the saved translations and rotations fromstep 1. In some embodiments, upon or after completing step 1 above, theactual proximity of the markers 4001, 4002, 4003 to their originalreference data-frame is dictated by marker noise and rigid bodyrigidity. In some embodiments, the markers will never overlay perfectlywith their counterparts that were stored when establishing the referenceframe. In some embodiments, the disclosed method can permit the markers4001, 4002, 4003 to get as close as possible to their original relativespacing.

FIGS. 39A-F illustrate examples of tracking methodology based on anarray of three attached markers 4001, 4002, and 4003 in accordance withsome embodiments of the invention. In some embodiments, the goal can bemarker 4001 on the origin, marker 4002 on the positive x-axis, andmarker 4003 in the x-y plan in a positive y direction (shown in FIG.39A). Assuming a starting configuration as shown in FIG. 39B, in someembodiments, the method can include translating the rigid body so thatmarker 4001 is at the origin as shown in FIG. 39C. In some embodiments,the method can then include rotation about the y-axis so that marker4002 is in the x-y plane (i.e., z=0) (see FIG. 39D). In someembodiments, the method can then include rotating the z-axis so thatmarker 4002 is at y=0, x coordinate positive (as shown in FIG. 39E).Finally, in some embodiments, the method can include rotating about thex-axis so that marker 4003 is at z=0, y coordinate positive. In someembodiments, a record of the translations and rotations can be retainedin order to utilize the negative values to transform position(s) backafter adding the point of interest. In some embodiments, whentranslating the rigid body so that the marker 4001 moves to the origin,the vector to add to each marker's position vector is simply thenegative 4001 position vector. In some embodiments, to determine thevalues of 0 to plug into the rotation matrices in steps 2, 3, and 4, usethe arctangent. For example, rotate marker 4002 about the y-axis to z=0where:

${M2} = \begin{bmatrix}4 \\5 \\6\end{bmatrix}$

FIG. 40 illustrates an example of a two dimensional representation forrotation about the y-axis in accordance with some embodiments of theinvention. As shown, FIG. 40 illustrates one embodiment showing atwo-dimensional representation looking down the axis about whichrotation occurs (e.g., y-axis is going into the page). In this example,a position rotation of 0=56.3° about the y-axis is needed to bring 4002to z=0. It should be appreciated that the appropriate direction (+ or −)of the rotation angle to plug into the rotation matrix can be confusing.In some embodiments, it is beneficial to draw the plane of the rotationwith the rotation axis coming out of the plane contained in the pagesurface (for example, the right-hand rule can provide a suitableorientation), then a counterclockwise rotation is positive and aclockwise rotation is negative. In the foregoing example, the axis wasgoing into the page surface, thus a clockwise rotation was positive.

FIGS. 41A-C illustrates an alternative representation of two dimensionalrepresentations for rotations about the axis, depicting how each planecan be drawn for counterclockwise positive and clockwise negative inaccordance with some embodiments of the invention. As shown, FIG. 41Aillustrates an alternative representation of a two dimensionalrepresentation for rotation about an X-axis. FIG. 41B illustrates analternative representation of a two dimensional representation forrotation about a Y-axis. FIG. 41C illustrates an alternativerepresentation of a two dimensional representation for rotation about aZ-axis

In some embodiments, to rotate the rigid body about the y-axis so that4002 is in the x-y plane (z=0):

$\theta_{y} = {+ {\tan^{- 1}\left( \frac{4002_{z}}{4002_{x}} \right)}}$

In some embodiments, to rotate the rigid body about the z-axis so that4002 is at y=0,

$\theta_{z} = {- {\tan^{- 1}\left( \frac{4002_{y}}{4002_{x}} \right)}}$

In some embodiments, to rotate the rigid body about the x-axis so that4003 is at z=0:

$\theta_{x} = {- {\tan^{- 1}\left( \frac{4003_{z}}{4003_{y}} \right)}}$

As described herein, the example method to transform markers 4001, 4002,4003 as close as possible to the reference frame can comprise; 1).translate the rigid body so that 4001 is at the origin (0, 0, 0), and2). rotate about the y-axis so that 4002 is in the x-y plane (i.e.,z=0), and 3). rotate about the z-axis so that 4002 is at y=0, xcoordinate positive, and 4). rotate about the x-axis so that 4003 is atz=0, y coordinate positive. In other embodiments, a method to reach thesame reference can comprise: 1). translate the rigid body so that 4001is at the origin (0, 0, 0), and 2). rotate about the x-axis so that 4002is in the x-y plane (i.e., z=0), and 3). rotate about the z-axis so that4002 is at y=0, x coordinate positive, 4). rotate about the x-axis sothat 4003 is at z=0, y coordinate positive. It should be appreciatedthat there are other possible methods and related actions, both in thereference frame chosen and in how the rigid body is manipulated to getit there. The described method is simple, but does not treat markersequally. The reference frame requires 4001 to be restricted the most(forced to a point), 4002 less (forced to a line), and 4003 the least(forced to a plane). As a result, errors from noise in markers aremanifested asymmetrically. For example, consider a case where in acertain frame of data, noise causes each of the three markers to appearfarther outward than they actually are or were (represented by 4001 a,4002 a, and 4003 a) when the reference frame was stored (as depicted inFIG. 42.)

In some embodiments, when the transformations are done to align theapparent markers “as close as possible” to their stored referenceposition, they will be offset. For example, when the stored point ofinterest is added, it will be misplaced in a direction on which markerwas chosen as 4001 in the algorithm (see 4003, 4003 a and 4002, 4002 afor example in FIG. 43).

Some embodiments provide additional or alternative methods for trackingpoints of interest that can involve more symmetrical ways of overlayingthe actual marker positions with the stored reference positions. Forexample, in some embodiments, for three markers 4001, 4002, 4003, atwo-dimensional fitting method typically utilized in zoology can beimplemented. (See, e.g., Sneath P. H. A., “Trend-surface analysis oftransformation grids,” J. Zoology 151, 65-122 (1967)). The method caninclude a least squares fitting algorithm for establishing a referenceframe and transforming markers to lie as close as possible to thereference. In this case, the reference frame is the same as describedearlier except that the common mean point (hereinafter referred to as“CMP”) is at the origin instead of marker 4001. In some embodiments, theCMP after forcing the markers into the x-y plane is defined in thefollowing equation (and can be represented in FIG. 44):

${CMP} = {\begin{bmatrix}\overset{\_}{x} \\\overset{\_}{y} \\0\end{bmatrix} = \begin{bmatrix}{\left( {{M\; 1_{x}} + {M\; 2_{x}} + {M\; 3_{x}}} \right)/3} \\{\left( {{M\; 1_{y}} + {M\; 2_{y}} + {M\; 3_{y}}} \right)/3} \\0\end{bmatrix}}$

In some embodiments, for the markers to be centered around CMP, themarkers can be translated by subtracting the CMP from 4001, 4002, and4003. It should be noted that the point of interest being tracked is notincluded in determining CMP_(ref).

In some embodiments, the method to transform markers as close aspossible to this reference frame can comprise; 1). translating the rigidbody so that 4001 is at the origin (0, 0, 0), and 2). rotating about they-axis so that 4002 is in the x-y plane (i.e., z=0), and 3). rotatingabout the z-axis so that 4002 is at y=0, x coordinate positive into thex-ray plane, and 4). rotating about the x-axis so that 4003 is at z=0, ycoordinate positive, and finally 5). calculate the CMP for the markers4001, 4002, 4003 and translating the rigid body so that the CMP is atthe origin (i.e., subtract the CMP from each point transformed). In someembodiments, steps 1-5 are done for the original set of markers forwhich the position of the point of interest was known and for the newset for which you are adding the point of interest. A further step canbe included for the new set, for example, 6). rotate about the z-axis tobest overlay the stored reference markers. In some embodiments, therotation angle θ is found using the formula from Sneath:

${\tan \mspace{11mu} \theta} = \frac{{\sum{x_{{pos}\; 2}y_{ref}}} - {\sum{x_{ref}y_{{pos}\; 2}}}}{{\sum{x_{ref}x_{{pos}\; 2}}} + {\sum{y_{ref}y_{{pos}\; 2}}}}$

In some embodiments, if M1, M2, M3 denote the stored reference markersand M′1, M′2, M′3 denote the position being tracked in this data-frame,the equation can be written:

$\tan^{- 1} = \left\lbrack \frac{\begin{matrix}{\left( {{M^{\prime}1_{x}M\; 1_{y}} + {M^{\prime}2_{x}M\; 2_{y}} + {M^{\prime}3_{x}M\; 3_{y}}} \right) -} \\\left( {{M\; 1_{x}M^{\prime}1_{y}} + {M\; 2_{x}M^{\prime}2_{y}} + {M\; 3_{x}M^{\prime}3_{y}}} \right)\end{matrix}}{\begin{matrix}{\left( {{M\; 1_{x}M^{\prime}1_{x}} + {M\; 2_{x}M^{\prime}2_{x}} + {M\; 3_{x}M^{\prime}3_{x}}} \right) +} \\\left( {{M\; 1_{y}M^{\prime}1_{y}} + {M\; 2_{y}M^{\prime}2_{y}} + {M\; 3_{y}M^{\prime}3_{y}}} \right)\end{matrix}} \right\rbrack$

It should be noted that this rotation angle can be small (e.g., smallerthan about 1°). In some embodiments, after the markers 4001, 4002, 4003are overlaid, some embodiments of the invention can include adding thepoint of interest then transforming the point of interest back to itstrue present location in the current frame of data. In some embodiments,to transform back, negative values saved from the forward transformationsteps 1-6 as discussed above can be utilized. That is, for instance, gofrom step 6 to step 5 by rotating by negative 0, go from step 5 to step4 by adding the CMP, etc.)

In some embodiments, using this least-squares algorithm, noise ismanifested more symmetrically and the point of interest will probably becalculated to be closer to its actual location. This can be illustratedin FIG. 45, which illustrates a depiction of results of applying a leastsquares fitting algorithm for establishing a reference frame andtransforming markers 4001, 4002, 4003 as shown in FIG. 44 includingnoise. Regardless of which method is used, it can be beneficial tomonitor the error between the marker locations at a given frame of dataand the reference marker locations. In some embodiments, the method caninclude calculating and sum the vector distances of each marker andreport the value in mm.

FIG. 46 for example illustrates a depiction of error calculation forreference frame markers in accordance with some embodiments of theinvention. In some embodiments, by continuously displaying this errorvalue, the agent can be alerted if markers 4001, 4002, 4003 have becomepartially obscured, or if a marker 4001, 4002, 4003 is no longersecurely or rigidly attached to the rigid body. In some embodiments,when performing a best fit on more than 3 markers, they cannot be forcedinto a plane, and therefore the problem becomes much more difficult. Insome embodiments, one solution is to inspect all or nearly all possibletriangles formed by groups of 3 markers 4001, 4002, 4003 and evaluatewhich one gives the least standard deviation of the angles of thevertices. See Chèze L, Fregly B. J., Dimnet J, Technical note: Asolidification procedure to facilitate kinematic analyses based on videosystem data,” Journal of Biomechanics 28(7), 879-884 (1995). In someother embodiments, the method can include calculating a least squaresfit of the vertices of the geometric shape, requiring iteration toperform matrix decomposition (i.e., Newton-Raphson method). For example,see Veldpaus F E, Woltring H J, Dortmans L J M G, ‘A least-squaresalgorithm for the equiform transformation from spatial markerco-ordinates’, Journal of Biomechanics 21(1), 45-54 (1988).

In some embodiments, when tracking 3D movement of a rigid body (forexample, a robot 15 end-effectuator 30 or a targeted bone) using anarray of 3 tracking markers 4001, 4002, 4003 that are rigidly attachedto the rigid body, one example method for quantifying motion can includedetermining the transformations (translation and rotations) for themovement from a first (neutral) position (defined here as “A”) to second(current frame) position (herein referred to as “B”). In someembodiments, it may be convenient to describe the rotations as a threeby three orientation matrix (direction cosines) of the rigid body in theposition B, and to treat the three translation values as a 3×1 vectorcontaining the x, y, z coordinates of the origin of the position Acoordinate system transformed to position B. In some embodiments, thedirection cosine matrix is a 3×3 matrix, the columns of which containunit vectors that originally were aligned with the x, y, and z axes,respectively, of the neutral coordinate system. In some embodiments, tobuild a direction cosine matrix, a 3×3 matrix, A, can be defined in amanner that its columns are unit vectors, i, j, and k, aligned with thex, y, and z axes, respectively:

$A = {\begin{bmatrix}i_{x} & j_{x} & k_{x} \\i_{y} & j_{y} & k_{y} \\i_{z} & j_{z} & k_{z}\end{bmatrix} = \begin{bmatrix}1 & 0 & 0 \\0 & 1 & 0 \\0 & 0 & 1\end{bmatrix}}$

Upon or after rotations of the coordinate system occur, in someembodiments, the new matrix (which is the direction cosine matrix, A′)is as follows, where the unit vectors i′, j′, and k′ represent the neworientations of the unit vectors that were initially aligned with thecoordinate axes:

$A^{\prime} = \begin{bmatrix}i_{x}^{\prime} & j_{x}^{\prime} & k_{x}^{\prime} \\i_{y}^{\prime} & j_{y}^{\prime} & k_{y}^{\prime} \\i_{z}^{\prime} & j_{z}^{\prime} & k_{z}^{\prime}\end{bmatrix}$

In some embodiments, to determine the direction cosines and translationvector, the origin and unit vectors can be treated as aligned with thecoordinate axes as four tracked points of interest in the mannerdescribed herein. For example, if the origin (o) and three unit vectors(i, j, k) are aligned with the coordinate axes, they are treated asvirtual tracked points of interest with coordinates of:

$o = {\left. \begin{bmatrix}0 \\0 \\0\end{bmatrix}\rightarrow i \right. = {\left. \begin{bmatrix}1 \\0 \\0\end{bmatrix}\rightarrow j \right. = {\left. \begin{bmatrix}0 \\1 \\0\end{bmatrix}\rightarrow k \right. = \begin{bmatrix}0 \\0 \\1\end{bmatrix}}}}$

In some embodiments, these points of interest can provide the directioncosines and translation for the movement when moved along with the threemarkers from position A to position B. In some embodiments, it may beconvenient to implement (for example execute) the method for moving thevirtual points to these four points placed into a 3×4 matrix, P.

In some embodiments, the matrix is as follows in position A:

$P = {\left\lbrack {\begin{matrix}o_{x} \\o_{y} \\o_{z}\end{matrix}\begin{matrix}i_{x} & j_{x} & k_{x} \\i_{y} & j_{y} & k_{y} \\i_{z} & j_{z} & k_{z}\end{matrix}} \right\rbrack = \begin{bmatrix}0 & 1 & 0 & 0 \\0 & 0 & 1 & 0 \\0 & 0 & 0 & 1\end{bmatrix}}$

In some embodiments, the matrix is as follows in position B:

$P^{\prime} = {\left\lbrack {\begin{matrix}a_{x} \\a_{y} \\a_{z}\end{matrix}\begin{matrix}b_{x} & c_{x} & d_{x} \\b_{y} & c_{y} & d_{y} \\b_{z} & c_{z} & d_{z}\end{matrix}} \right\rbrack = \begin{bmatrix}o_{x}^{\prime} & {i_{x}^{\prime} + o_{x}^{\prime}} & {j_{x}^{\prime} + o_{x}^{\prime}} & {k_{x}^{\prime} + o_{x}^{\prime}} \\o_{y}^{\prime} & {i_{y}^{\prime} + o_{y}^{\prime}} & {j_{y}^{\prime} + o_{y}^{\prime}} & {k_{y}^{\prime} + o_{y}^{\prime}} \\o_{z}^{\prime} & {i_{z}^{\prime} + o_{z}^{\prime}} & {j_{z}^{\prime} + o_{z}^{\prime}} & {k_{z}^{\prime} + o_{z}^{\prime}}\end{bmatrix}}$

In some embodiments, after movement, the direction cosine matrix is

$A^{\prime} = {\begin{bmatrix}i_{x}^{\prime} & j_{x}^{\prime} & k_{x}^{\prime} \\i_{y}^{\prime} & j_{y}^{\prime} & k_{y}^{\prime} \\i_{z}^{\prime} & j_{z}^{\prime} & k_{z}^{\prime}\end{bmatrix} = \begin{bmatrix}{b_{x} - a_{x}} & {c_{x} - a_{x}} & {d_{x} - a_{x}} \\{b_{y} - a_{y}} & {c_{y} - a_{y}} & {d_{y} - a_{y}} \\{b_{z} - a_{z}} & {c_{z} - a_{z}} & {d_{z} - a_{z}}\end{bmatrix}}$

In some embodiments, the vector o′ represents the new position of theorigin. In some embodiments, after moving the three markers 4001, 4002,4003 from position A to position B, and bringing the four points (as a3×4 matrix) along with the three markers 4001, 4002, 4003, thetranslation of the origin is described by the first column. Further, insome embodiments, the new angular orientation of the axes can beobtained by subtracting the origin from the 2^(nd), 3^(rd), and 4^(th)columns. These methods should be readily apparent from the followinggraphic representation in FIG. 47, which illustrates a graphicalrepresentation of methods of tracking three dimensional movement of arigid body.

In some embodiments, if more than three markers 4001, 4002, 4003 areutilized for tracking the movement of a rigid body, the same method canbe implemented repeatedly for as many triads of markers as are present.For example, in a scenario in which four markers, M1, M2, M3, and M4,are attached to the rigid body, there can be four triads: those formedby {M1, M2, M3}, {M1, M2, M4}, {M1, M3, M4}, and {M2, M3, M4}. In someembodiments, each of these triads can be used independently in themethod described hereinbefore in order to calculate the rigid bodymotion. In some embodiments, the final values of the translations androtations can then be the average of the values determined using thefour triads. In some embodiments, in the alternative or in addition,other methods for achieving a best fit when using more than 3 markersmay be used.

In some embodiments, when tracking with four markers, in a scenario inwhich one of the four markers becomes obscured, it can desirable toswitch to tracking the rigid body with the remaining three markersinstead of four. However, this change in tracking modality can cause asudden variation in the results of one or more calculations utilized fortracking. In some embodiments, the variation can occur because thesolution from the one remaining triad may be substantially differentthan the average of 4 triads. In some embodiments, if using the trackedposition of the rigid body in a feedback loop to control the position ofa robot 15 end-effectuator, the sudden variation in results of thecalculation can be manifested as a physical sudden shift in the positionof the robot 15 end-effectuator 30. In some embodiments, this behavioris undesirable because the robot 15 is intended to hold a guide tube 50steady with very high accuracy.

Some embodiments include an example method for addressing the issue ofsudden variation that occurs when one of the four markers M1, M2, M3, M4is blocked, thereby causing the position to be calculated from a singletriad instead of the average of four triads, can include reconstructingthe blocked marker as a virtual marker. In some embodiments, toimplement such reconstructing step with high accuracy, the most recentframe of data in which all four markers M1, M2, M3, M4 are visible canbe retained substantially continuously or nearly continuously (forexample in a memory of a computing device implementing the subjectexample method). In some embodiments, if all four markers M1, M2, M3, M4are in view, the x-axis, y-axis, and z-axis coordinates of each of thefour markers M1, M2, M3, and M4 are stored in computer 100 memory. Itshould be appreciated that in some embodiments, it may unnecessary tolog all or substantially all frames and is sufficient to overwrite thesame memory block with the most recent marker coordinates from a fullvisible frame. Then, in some embodiments, at a frame of data in whichone of the four markers M1, M2, M3, and M4 is lost, the lost marker'sposition can be calculated based on the remaining triad, using theexample method described herein for remaining three markers. That is,the triad (the three visible markers) is transformed to a reference. Thestored set of markers is then transformed to the same reference usingthe corresponding triad with the fourth marker now acting as a virtuallandmark. The recovered position of the lost fourth marker can then betransformed back to the current position in space using the inverse ofthe transformations that took it to the reference position. In someembodiments, after the lost marker's position is reconstructed,calculation of the rigid body movement can be performed as before, basedon the average of the fourth triads, or other best fit method fortransforming the rigid body from position A to position B.

In some embodiments, an extension to the methods for reconstructingmarkers 720 is to use multiple ambiguous synchronized lines of sight viamultiple cameras 8200 tracking the same markers 720. For example, two ormore cameras 8200 (such as Optotrak® or Polaris®) could be set up fromdifferent perspectives focused on the tracking markers 720 on thetargeting fixture 690 or robot 15. In some embodiments, one camera unitcould be placed at the foot of a patient's bed, and another could beattached to the robot 15. In some embodiments, another camera unit couldbe mounted to the ceiling. In some embodiments, when all cameras 8200substantially simultaneously view the markers 720, coordinates could betransformed to a common coordinate system, and the position of any ofthe markers 720 would be considered to be the average (mean) of thatmarker's three dimensional position from all cameras used. In someembodiments, even with extremely accurate cameras, an average is neededbecause with system noise, the coordinates as perceived from differentcameras would not be exactly equal. However, when one line of sight isobscured, the lines of sight from other cameras 8200 (where markers 720can still be viewed) could be used to track the robot 15 and targetingfixture 690. In some embodiments, to mitigate twitching movements of therobot 15 when one line of sight is lost, it is possible that the marker720 positions from the obscured line of sight could be reconstructedusing methods as previously described based on an assumed fixedrelationship between the last stored positions of the markers 720relative to the unobstructed lines of sight. Further, in someembodiments, at every frame, the position of a marker 720 from camera 1relative to its position from camera 2 would be stored; then if camera 1is obstructed, and until the line of sight is restored, this relativeposition is recalled from computer memory (for example in memory of acomputer platform 3400) and a reconstruction of the marker 720 fromcamera 1 would be inserted based on the recorded position of the markerfrom camera 2. In some embodiments, the method could compensate fortemporary obstructions of line of sight such as a person standing orwalking in front of one camera unit.

In certain embodiments, when a marker M1, M2, M3, M4 is lost but issuccessfully reconstructed in accordance with one or more aspectdescribed herein, the marker that has been reconstructed can be renderedin a display device 3411. In one example implementation, circlesrepresenting each marker can be rendered graphically, coloring thecircles for markers M1, M2, M3, M4 that are successfully tracked ingreen, markers M1, M2, M3, M4 that are successfully reconstructed inblue, and markers M1, M2, M3, M4 that cannot be tracked or reconstructedin red. It should be appreciated that such warning for the agent canserve to indicate that conditions are not optimal for tracking and thatit is prudent to make an effort for all four tracking markers to be madefully visible, for example, by repositioning the cameras or standing ina different position where the marker is not blocked. Other formatsand/or indicia can be utilized to render a virtual marker and/ordistinguish such marker from successfully tracked markers. In someembodiments, it is possible to extend the method described herein tosituations relying on more than four markers. For example, inembodiments in which five markers are utilized on one rigid body, andone of the five markers is blocked, it is possible to reconstruct theblocked marker from the average of the four remaining triads or fromanother method for best fit of the 4 remaining markers on the storedlast visible position of all 5 markers. In some embodiments, oncereconstructed, the average position of the rigid body is calculated fromthe average of the 10 possible triads, {M1, M2, M3}, {M1, M2, M4}, {M1,M2, M5}, {M1, M3, M4}, {M1, M3, M5}, {M1, M4, M5}, {M2, M3, M4}, {M2,M3, M5}, {M2, M4, M5}, and {M3, M4, M5} or from another method for bestfit of 5 markers from position A to position B.

As discussed above, in some embodiments, the end-effectuator 30 can beoperatively coupled to the surgical instrument 35. This operativecoupling can be accomplished in a wide variety of manners using a widevariety of structures. In some embodiments, a bayonet mount 5000 is usedto removably couple the surgical instrument 35 to the end-effectuator 30as shown in FIG. 48. For example, FIG. 48 shows a perspective viewillustrating a bayonet mount 5000 used to removably couple the surgicalinstrument 35 to the end-effectuator 30. In some embodiments, thebayonet mount 5000 securely holds the surgical instrument 35 in placewith respect to the end-effectuator 30, enabling repeatable andpredictable location of operational edges or tips of the surgicalinstrument 35.

In some embodiments, the bayonet mount 5000 can include ramps 5010 whichallow identification of the surgical instrument 35 and ensure compatibleconnections as well. In some embodiments, the ramps 5010 can be sizedconsistently or differently around a circumference of the bayonet mount5000 coupled to or integral with the surgical instrument 35. In someembodiments, the differently sized ramps 5010 can engage complementaryslots 5020 coupled to or integral with the end-effectuator 30 as shownin FIG. 48.

In some embodiments, different surgical instruments 35 can includedifferent ramps 5010 and complementary slots 5020 to uniquely identifythe particular surgical instrument 35 being installed. Additionally, insome embodiments, the different ramps 5010 and slots 5020 configurationscan help ensure that only the correct surgical instruments 35 areinstalled for a particular procedure.

In some embodiments, conventional axial projections (such as those shownin U.S. Pat. No. 6,949,189 which is incorporated herein as needed toshow details of the interface) can be mounted to or adjacent the ramps5010 in order to provide automatic identification of the surgicalinstruments 35. In some embodiments, other additional structures can bemounted to or adjacent the ramps 5010 in order to provide automaticidentification of the surgical instruments 35. In some embodiments, theaxial projections can contact microswitches or a wide variety of otherconventional proximity sensors in order to communicate the identity ofthe particular surgical instrument 35 to the computing device 3401 orother desired user interface. Alternatively, in some other embodimentsof the invention, the identity of the particular surgical instrument 35can be entered manually into the computing device 3401 or other desireduser interface.

In some embodiments, instead of a targeting fixture 690 consisting of acombination of radio-opaque 730 and active markers 720, it is possibleto register the targeting fixture 690 through an intermediatecalibration. For example, in some embodiments, an example of such acalibration method could include attaching a temporary rigid plate 780that contains radio-opaque markers 730, open mounts 785 (such as snaps,magnets, Velcro, or other features) to which active markers 720 canlater be attached in a known position. For example, see FIGS. 49A-Fwhich depict illustrations of targeting fixtures 690 coupled to a spineportion 19 of a patient 18 in accordance with one embodiment of theinvention). The method can then include scanning the subject (using forexample CT, MRI, etc.), followed by attaching a percutaneous tracker 795such as those described earlier or other array of 3 or more activemarkers 720 rigidly affixed to the anatomy 19 as for example in FIG.49B, and then attaching active markers 720 to the temporary plate 780 inthe known positions dictated by the snaps, magnets, velcro, etc., asillustrated in FIG. 49C. In some embodiments, a further step can includeactivating the cameras 8200 to read the position of the tracker 795rigidly affixed to the anatomy 19 at the same time as the active markers720 on the temporary plate 780. This step establishes the position ofthe active markers 720 on the temporary plate 780 relative to theradio-opaque markers 730 on the temporary plate 780 as well as thepositions of the active markers 720 on the tracker 795 relative to theactive markers 720 on the temporary plate 780, and therefore establishesthe position of the anatomy relative to the active markers 720 on thetracker 795. The temporary plate 780 can be removed (as illustrated inFIG. 49D), including the active markers 720 and radio-opaque markers730. These markers are no longer needed because registration has beenperformed relative to the active markers on the rigidly affixed tracker795.

In some alternative embodiments, variants of the order of the abovedescribed steps may also be used. For instance, the active markers 720could already be attached at the time of the scan. This method hasadvantage that the radio-opaque markers 730 can be positioned close tothe anatomy of interest without concern about how they are attached tothe tracker 795 with active markers 720. However, it has thedisadvantage that an extra step is required in the registration process.In some embodiments, a variant of this method can also be used forimproved accuracy in which two trackers of active markers 720 areattached above and below the region of interest. For example, a trackerrostral to the region of interest (shown as 795) could be a spinousprocess 2310 clamp in the upper lumbar spine and a tracker caudal to theregion of interest (shown as 800) could be a rigid array of activemarkers 720 screwed into the sacrum (see for example FIG. 49E). Aftercalibration, the temporary plate 780 is removed and the area between thetwo trackers (within the region 805) is registered (see for example FIG.49F).

Some embodiments can include methods for transferring registration. Forexample, a registration performed to establish the transformations inorder to transpose from a medical image coordinate system (such as theCT-scanned spine) to the coordinate system of the cameras, can later betransferred to a different reference. In the example described in theabove related to FIGS. 49A-F, a temporary fixture 780 with radio-opaquemarkers 730 and active markers 720 is placed on the patient 18 andregistered. Then, a different fixture 795 is attached to the patientwith active markers 720 only. Then the cameras (for example, camera8200) are activated, and the active markers 720 on the temporary plate780 are viewed simultaneously with the active markers 720 on the newtracking fixture 795. The necessary transformations to get from thetemporary markers (those on the temporary plate 780) to the new markers(i.e. the markers on fixture 795) are established, after which thetemporary plate 780 can be removed. In other words, the registration wastransferred to a new reference (fixture 795). In some embodiments, itshould be possible to repeat this transferal any number of times.Importantly, in some embodiments, one registration can also beduplicated and transferred to multiple references. In some embodiments,transferal of registration to multiple references would provide a meansfor tracking relative motion of two rigid bodies. For example, atemporary targeting fixture may be used to register the anatomy to thecameras 8200. Then, two new targeting fixtures may be placed on separatebones that are both included in the medical image used for registration.If the registration from the temporary targeting fixture is transferredto both of the new targeting fixtures, both of these bones may betracked simultaneously, and the position of the robot end effectuator 30or any other tracked probe or tool relative to both bones can bevisualized. If one bone then moves relative to the other, the endeffectuator's position would be located differently relative to the twotrackers and the two medical images (for example, see FIGS. 50B-50Dshowing the two trackers 796 and 797 positioned on a portion of spine19).

In some embodiments, after registration is transferred to both trackers796, 797, the robot end effectuator 30 may be perceived by both trackers796, 797 to be positioned as shown. In some embodiments, it is possiblethat one of the bones to which a tracker is mounted moves relative tothe other, as shown in exaggerated fashion in FIG. 50C. In someembodiments, if the end effectuator 30 is considered fixed, theperception by the tracking system 3417 and software would be that thespine 19 was positioned in two possible relative locations, depending onwhich tracker is followed (see for example, the representation in FIG.50D). Therefore, in some embodiments, by overlaying representations ofboth medical images, it becomes possible to visualize the relativemovement of the bones on which the trackers 796, 797 are attached. Forexample, instead of displaying the re-sliced medical image on the screenand showing the position of the robot end effectuator 30 relative tothat image, two re-sliced medical images could be overlapped (eachallowing some transparency) and simultaneously displayed, showing oneposition where the robot end effectuator currently is positionedrelative to both images (see FIGS. 50E-50F). However, the duplication ofbones would make the representation cluttered, and therefore in someembodiments, it can be possible to automatically or manually segment themedical image such that only bones that do not move relative to aparticular tracker 796, 797 are represented on the image (shown in FIG.50E with the bones 19 a highlighted as in relation to bone regionsmeaningful to tracker 796 with regions 19 b faded, and FIG. 50F with thebones 19 a highlighted in relation to bone regions meaningful to tracker797, with regions 19 b faded). Segmenting would mean hiding, fading, orcropping out the portion of the 3D medical image volume that the userdoes not want to see (represented as the faded regions 19 b in FIGS. 50Eand 50F).

In some embodiments, segmentation could involve identifying borderingwalls on the 3D image volume or bordering curves on 2D slices comprisingthe medical image. In some embodiments, by segmenting simple six-sidedvolumes, enough separation of critical elements could be visualized forthe task. In some embodiments, bones on the slice from a CT scansdepicted in FIGS. 50G and 50H are shown with segmentation into region 20a, corresponding to the bone regions 19 a referred to in FIGS. 50E and50F, and 20 b, corresponding to region 19 b. In some embodiments, theregions 20 a and 20 b can be represented in different shades of color(for example, blue for 20 a and yellow for 20 b). Furthermore, as shown,the segmentation as displayed is depicted to proceed in and out of thepage to include the entire CT volume. Moreover, although it goes rightthrough the disc space, this segmentation cuts through one spinousprocess 2310 in the image in FIG. 50G, and does not follow the facetjoint articulations to segment independently moving bones, as shown in adifferent slice represented in FIG. 50H. However, the re-sliced imagesof these overlapped volumes should still be useful when placing, forexample, pedicle screws since the pedicles are properly segmented in theimages.

An example of transferal of registration to multiple trackers includesconventional pedicle screw placement followed by compression ordistraction of the vertebrae. For example, if pedicle screws are beingplaced at lumbar vertebrae L4 and L5, a tracker could be placed on L3and registered. In some embodiments, conventional pedicle screws couldthen be placed at L4 and L5, with extensions coming off of each screwhead remaining after placement. In some embodiments, two new trackers(for example, trackers substantially similar to 796, 797) could then beattached to the extensions on the screw heads, one at L4 and one at L5.Then, the registration could be transferred to both of these newtrackers and a tracker at L3 could be removed or thereafter ignored. Insome embodiments, if the medical image is segmented so that L4 androstral anatomy is shown relative to the tracker on L4 (while L5 andcaudal anatomy is shown relative to the tracker on L5), then it can bepossible to see how the L4 and L5 vertebrae move relative to oneanother, as compressive or distractive forces are applied across thatjoint. In some embodiments, such compression or distraction might beapplied by the surgeon when preparing the disc space for an inter-bodyspacer, or inserting the spacer, or when compressing the vertebraetogether using a surgical tool after the inter-body spacer is in place,and before locking the pedicle screw interconnecting rod.

In some embodiments, if there is snaking of the spine, for example, whenconventional screws are driven in place or the surgeon applies a focalforce on one portion of the spine, the two marker trees will move(illustrated as 795 a for tracker 795 and 800 a for tracker 800) bydifferent amounts and to different orientations (illustrated in FIG.50A). The altered orientations and positions of the trackers 795, 800can be used to calculate how the spine has snaked and adjust theperceived position of the robot 15 or probe to compensate. In someembodiments, because there are multiple degrees of freedom of thevertebrae, knowledge of how the two trackers' orientations shift doesnot allow a single unique solution. However, it can be assumed that thebending is symmetrical among all the vertebrae to calculate the newposition, and even if this assumption is not perfect, it should providea reasonably accurate solution. In some embodiments, experimentstracking how cadaveric spines respond to focal forces can be used tocollect data that will help to predict how the two ends of the lumbarspine would respond during particular types of external loading.

In some embodiments, it is possible to use the same surgical robot 15already described for navigation with 3D imaging in a different settingwhere only 2 fluoroscopic views are obtained. In this instance, thesurgical robot 15 will be able to accurately move to a desired positionthat is pre-planned on these two fluoroscopic views. Since the twofluoroscopic views can represent views to which the surgeon orradiologist is already accustomed, planning trajectories on these viewsshould be straightforward. In obtaining the fluoroscopic views, a methodis needed to establish the position of the coordinate system of theanatomy relative to the robot's 15 coordinate system. In someembodiments, a way to fulfill this registration is to obtain thefluoroscopic views while a targeting fixture 690 that includes featuresthat are identifiable on the fluoroscopic images is attached to thepatient 18. For example, FIG. 51 shows an example of a fixture for usewith fluoroscopic views in accordance with one embodiment of theinvention. In some embodiments, the targeting fixture 690 as shown caninclude features that will appear on 2 fluoroscopic views and activemarkers 720 for real-time tracking. This targeting fixture 690 hasproperties that will aid in the ability to set up the coordinate systemof the anatomy from the two fluoroscopic images. For example, in someembodiments, the posts 75 as shown are symmetrically spaced around theframe 700 so that posts 75 and/or their embedded markers 730 wouldoverlay on an x-ray image. That is, if there is no parallax, two posts75 in an aligned position would appear as a single line segment insteadof two, or two posts 75, each with two embedded radio-opaque markers730, would appear as two dots instead of four dots on an x-ray image.These features allow and facilitate the patient 18 or fluoroscopymachine's position to be adjusted until such overlapping is achieved.Similarly, from top or bottom view, the posts 75 and/or their embeddedmarkers 730 would overlap, with a single post appearing as a dot insteadof a line segment or two embedded markers 730 in one post appearing asone dot instead of two once the fluoroscopy machine and patient 18 areadjusted to be aligned as desired. In some embodiments, the posts 75 maybe designed to be temporarily inserted (i.e., they are present duringthe scan but are later unplugged from the frame during the procedure sothey are not in the way of the user). In some embodiments, the activemarkers 720 are necessary for later tracking but do not necessarily needto be present during the scan as long as they can be attached withprecision to a known position on the frame 700 relative to theradio-opaque makers 730. For example, in some embodiments, conventionalsockets on the fixture 690 could later allow the active markers 720 tobe snapped in to a location dictated by the manufacturing of the frame700 or calibrated using a digitizing probe. Furthermore, note that thegoal is not necessarily to get perfect lateral and anteroposterioranatomical views of the spine or other anatomy. The goal is to getalignment of the fixture 700 on the x-ray view. Although it may bebeneficial in understanding what it being visualized to also achievealignment with the anatomical planes, it is unnecessary forregistration. An example of how the targeting fixture 690 might appearon anteroposterior or “A-P” and lateral x-rays when affixed to thepatient's back but not yet aligned with the x-ray projection is shown inFIGS. 52A-52B.

In some embodiments, after adjusting the position of the patient 18 andfluoroscopy unit, an overlay with good certainty may be obtained forimages with radio-opaque markers 730. FIGS. 53A-B for exampleillustrates expected images on anteroposterior and lateral x-rays of thespine with a well aligned fluoroscopy (x-ray) machine in accordance withone embodiment of the invention. As shown, the fluoroscopic images donot need the frame 700 to be positioned exactly aligned with the anatomyor rotated to be vertical and horizontal. In some embodiments, thefluoroscopically obtained images are not required to have the correctaspect ratio such that the image properly represents a calibratedcoordinate system. In some embodiments, it is possible to rescale theimage to adjust the aspect ratio using known distances between posts 75or between markers 730, x-ray visible lengths of posts 75, or assumingthe image should be perfectly circular or square. These distances areknown in advance of obtaining the images by the manufacturing process,or by calibration using a digitizing probe or other means. In someembodiments, provided parallax is considered, the ratio of knowninter-marker distances can be compared to the ratio of inter-markerdistances measured on planar images and used to scale the planar imageto achieve the correct aspect ratio. In some embodiments, it is notnecessary to rescale the image, but it may help the user to bettervisualize the image and anatomy when it is displayed in the appropriateaspect ratio. In some embodiments, the comparison can also be used todetermine the number of pixels per mm on the image for use indetermining relative position of radio-opaque markers 730 and plannedtrajectory tip and tail. In some embodiments, rescaling facilitatesequations for mapping between 2D and 3D space because the pixels per mmin the x and y direction are the same value.

In some embodiments, after obtaining two images, the two images can beused to construct a 3D Cartesian coordinate system because theyrepresent images of the same thing (the fixture) from two orthogonalviews. For example, the A-P image could be used to represent the X-Zplane, and the lateral image could be used to represent the Y-Z plane.Radio-opaque markers 730 on the A-P image have known x-axis and z-axiscoordinates (as recorded from the manufacturing process or bycalibration using a digitizing probe or other means), and the sameradio-opaque markers 730 have known y-axis and z-axis coordinates on thelateral image. Therefore, in some embodiments, the x-axis, y-axis, andz-axis coordinates of the markers 730 can be found on the two images,and the positions of the anatomy and planned trajectories relative tothese reference points can be related to these reference positions. Insome embodiments, the mapping of a point from 3D space to the 2D imageand vice versa can be performed knowing the constant mm per pixel, C, oncoronal or sagittal images, and multiplying or dividing points by theseconstants if the center of the image and coordinate system have beenshifted to overlap.

FIGS. 54A-B illustrates expected images on anteroposterior and lateralx-rays of the spine with a well aligned fluoroscopy (x-ray) machine inaccordance with one embodiment of the invention. As shown, FIGS. 54A-Binclude overlaid computer-generated graphical images showing the plannedtrajectory (red 6001) and the current actual position of the robot 15end-effectuator 30 (light blue 6003). The red circle in 6001 is providedfor the user to identify the tail of the planned trajectory (as opposedto the tip). In other embodiments, the line segment could have differentcolored ends or different shapes on each end (pointed vs. blunt) fordistinguishing tip from tail.

In some embodiments, assuming the A-P x-ray represents the X-Z plane andthe lateral x-ray represents the Y-Z plane, the algorithm for planning atrajectory and relating this planned trajectory to the robot 15coordinate system can include the following steps; 1). Draw a line onthe A-P and lateral x-ray views representing where the desiredtrajectory should be positioned (see for example FIGS. 54A-54B). In someembodiments, the next step can include; 2). from the A-P view, find theX and Z coordinates of the reference opaque markers and of the tip andtail of the desired trajectory, and 3). from the lateral view, find theY and Z coordinates of the reference opaque markers and of the tip andtail of the desired trajectory, and 4). based on the known coordinatesof the active markers relative to the opaque markers, transform theX,Y,Z coordinates of the tip/tail into the coordinate system of theactive markers. In some embodiments, the method can include store thelocations of tip and tail in this coordinate system in computer 100memory for later retrieval. In some embodiments, the next steps of themethod can include; 5). at any frame in real time, retrieve the activemarker 720 locations in the coordinate system of the cameras 8200, and6). based on the stored coordinates of the tip and tail relative to theactive markers 720 and the current location of the active markers 720 inthe coordinate system of the cameras 8200, calculate the currentlocation of the desired tip and tail in the coordinate system of thecameras 8200. In some embodiments, the next steps of the method caninclude; 7). transform the active marker 720 locations and thetrajectory tip/tail locations into the coordinate system of the robot 15using methods described before in which markers on the robot 15 areutilized as references, and 8). send the robot 15 to the desiredtip/tail locations using methods described previously.

In some embodiments, while the robot 15 moves to position itself in thedesired orientation and position, it is possible to overlay a graphicalrepresentation of the current location of the robot 15 on thefluoroscopic images by a method that can include; 1). retrieve currentlocation of the robot 15 guide tube 50 in the coordinate system of thecameras 8200 based on active markers 720 attached to the robot, and 2).transform the guide tip and tail to the coordinate system of the medicalimages based on the locations of active markers on the targeting fixture690, and 3). represent the current positions of tip/tail of the guidetube 50 on the A-P image by a line segment (or other suitable graphicalrepresentation) connecting the X,Z coordinates of the tip to the X,Zcoordinates of the tail (see for example FIGS. 54A-B), and 4). representthe current positions of tip/tail of the guide tube 50 on the lateralimage by a line segment (or other suitable graphical representation)connecting the Y,Z coordinates of the tip to the Y,Z coordinates of thetail.

In some embodiments, in constructing the Cartesian coordinate systembased on the two images, it is important to consider directionality.That is, in some embodiments, an x-ray image of the X-Z plane could showpositive X to the right and negative X to the left or vice versa. Insome embodiments, it could show positive Z upward and negative Zdownward or vice versa. In some embodiments, an x-ray image of the Y-Zplane could show positive Y to the right and negative Y to the left orvice versa. In some embodiments, it could show positive Z upward andnegative Z downward or vice versa. In some embodiments, if an incorrectassumption is made about the directionality of one of the axes, it wouldmean that the constructed 3D coordinate system has one or more of itsaxes pointing in the wrong direction. In some embodiments, this may sendthe robot 15 to an incorrect position. In some embodiments, one way ofensuring the correct directionality is to query to the user requestingverification of directionality on the images and/or allowing them toflip (mirror) the images on the display 29, 150, 3401. In someembodiments, another way of ensuring the correct directionality is todesign the targeting fixture 690 so that the radio-opaque markers 730are spaced asymmetrically. In some other embodiments, another way ofensuring the correct directionality is to design the targeting fixture690 with additional radio-opaque features that unambiguously identifytop, bottom, left, right, front and rear on images. For example, FIGS.55A-55B illustrates expected images on anteroposterior and lateralx-rays of the spine with a well aligned fluoroscopy (x-ray) machine. Asshown, the targeting fixture 690 illustrated in FIGS. 55A-B includes afeature 755 designed to substantially eliminate ambiguity aboutdirectionality in accordance with one embodiment of the invention. Asshown, the feature 755 could reveal a “L”, “T”, or other symbol on thex-ray when the view is correct so as to substantially eliminatedirectional ambiguity. Furthermore, the “T” feature could be drawn inscript (e.g.,

) or other asymmetric letter or symbol used so that if an inverted ormirrored x-ray image is presented, the inverted nature is clear and canbe compensated.

In some embodiments, the algorithm described here provides the user withtwo perpendicular x-ray views from which to plan a trajectory, andprovides a visual feedback of the current location of a probe.Typically, these two views might be lateral and anteroposterior (A-P)views. In some embodiments, it might also be desirable for the user tosee a third plane (for example, an axial plane). Based on knowledge ofthe anatomy and landmarks visible on the x-rays, in some embodiments, itis possible to create a rough “cartoon” showing an axial view. In someembodiments, the cartoon may help the user understand the approximatecurrent location of the robot 15 or probe. FIG. 56 shows how such acartoon can be generated from the x-rays. Note that the cartoon will beimperfect with respect to details such as the curvature of the vertebralbody, but key landmarks such as the pedicle boundaries should bereasonably well defined. Such an approach would be based on how atypical vertebra is shaped. For example, FIG. 56 illustrates an axialview of a spine showing how a cartoonish axial approximation of thespine 5601 can be constructed based on a lateral x-ray 5602 and ananteroposterior x-ray 5603 in accordance with one embodiment of theinvention. As shown, locations where key landmarks on the adjacent x-rayviews intersect the cartoon can be identified with horizontal orvertical lines overlapping the cartoon and x-rays. The user positionsthese lines using a software interface or software automaticallyrecognizes these features on the x-rays so that the lines intersect thekey landmarks, such as a line just tangent to the vertebral body leftborder 5604, vertebral body right border 5605, vertebral body anteriorwall 5606, vertebral body posterior wall 5607, posterior spinal canal5608, left inner pedicle border 5609, left outer pedicle border 5610,tip of spinous process 5611, etc. After using the software to move theselines so that they intersect correct locations on the x-rays, thesoftware can then stretch and morph the cartoon as needed to fit theseanatomical limits. A view on the computer display of the axial planeshowing this cartoon and the planned trajectory and current position ofrobot or probe can be generated by the software to provide additionalvisual feedback for the user.

In some embodiments, in order to achieve well aligned x-rays like thoseshown in FIGS. 53A-B, one possible method is trial and error. Forexample, the user can try to get the x-ray machine aligned to thetargeting fixture 690, attempt to assess alignment by eye, then shoot anx-ray and see how it looks. In some embodiments, if dots are misaligned(for example as shown in FIGS. 52A-52B), adjustments would be made and anew x-ray image can be prepared. This method can be effective but can bedependent on the skill of the operator in assessing alignment and makingcorrections, and therefore can result in x-ray exposure to the patient18 and staff. In some embodiments, it is possible to create a tool toassist in the alignment of the targeting fixture 690. In someembodiments, the tool could be a conventional laser that can be attachedto the emitter or collector panel of the x-ray machine, capable ofpassing a laser beam parallel to the direction that the x-rays willtravel. In some embodiments, the laser could be attached temporarily(using a conventional magnet or adhesive) or permanently, connected toan arm extending from the x-ray machine and oriented in the correctdirection, enabling the directed beam to shine down toward the fixture690. In some embodiments, if the fixture 690 has a geometric,electronic, or other features capable of visual or other feedback to theuser regarding the vector direction of this laser light, it would allowalignment of the x-ray arm without taking any x-rays. An example of sucha feature is shown in FIG. 57A and FIG. 57B. FIGS. 57A-B illustratesexamples of targeting fixtures 690 that facilitate desired alignment ofthe targeting fixture 690 relative to the x-ray image plane inaccordance with one embodiment of the invention. As shown, someembodiments include a feature 765 temporarily added to the targetingfixture 690 In some embodiments, feature 765 facilitates desiredalignment of the targeting fixture 690 relative to the x-ray image planefrom an AP view when a laser (attached to the face of the x-ray emitteror collector) is directed through the opening 766 and toward thecrosshairs 767 at the base. In some embodiments, if the laser light doesnot strike the crosshairs 767 dead center, further adjustment of thex-ray unit's orientation is needed. The temporarily added feature 765that facilitates desired alignment of the targeting fixture 690 relativeto the x-ray image plane from a lateral view when a laser (attached tothe face of the x-ray emitter or collector) is directed through theopening and toward the crosshairs at the opposite face. In someembodiments, if the laser light does not strike the crosshairs deadcenter, further adjustment of the x-ray unit's orientation is needed.

In some embodiments, this method for aligning the radio-opaque markers730 would have the advantage over trial-and-error methods that areaffected by parallax effects, and as described below, do not confoundthe ability to align markers as needed. For example, with parallax, itmay not be clear to the user when good alignment of the markers 730 isachieved, depending on how symmetrically spaced the markers 730 areabout the center of the image.

With parallax error, the x-rays may not pass through the subject in astraight line and instead travel from emitter to receiver in a conicalpattern. This conical path can produce an image where the details ofanatomy on the 2D x-ray that are closer to the emitter of the x-rayswill appear farther apart laterally than details of the anatomy that arecloser to the receiver plate. In the case of x-ray images in FIGS.53A-B, instead of the radio-opaque markers 730 appearing overlaid, theymay appear as shown in FIGS. 58A-B. For example, FIGS. 58A-B illustratesexpected images on anteroposterior and lateral x-rays of the spine witha well aligned fluoroscopy (x-ray) machine when parallax is present inaccordance with one embodiment of the invention. As shown, parallaxaffects spacing symmetrically about the x, y center of the image, withlocations of markers 730 closer to the receiver plate of the x-ray unitappearing closer to the center of the image.

Further, in the description, two terms used are “near plane” and “farplane”—these terms refer to markers in the 2D views that appear fartherapart or closer together because of parallax. The reason markers arefarther apart or closer together is because of their proximity to theemitter or collector of the x-ray machine, with markers nearer theemitter appearing farther apart and markers nearer the collector closertogether. However, rather than referencing distance from emitter andcollector, “near plane” refers to markers that appear magnified (nearerto the eye) and “far plane” refers to markers that appear more distant.

Parallax will affect the image symmetrically about the center of theimage. For example, in some embodiments, two markers 730 (one in nearplane and one in far plane) that are in the same projected position, andare at the center of the image, may appear to be exactly on top of eachother, whereas markers 730 in the near plane and far plane that are inthe same projected position, but are close to the edge of the image mayappear separated by a substantial distance.

FIG. 59A illustrates two parallel plates with identically positionedradio-opaque markers 730 in accordance with one embodiment of theinvention. As shown, this illustrates possible marker 730 separation onan x-ray from markers 730 on two plates that are in the same projectedline of sight. FIG. 59B illustrates resulting expected x-raydemonstrating how marker overlay is affected due to parallax using thetwo parallel plates as shown in FIG. 59A in accordance with oneembodiment of the invention. By comparing the two parallel plates withidentically positioned radio-opaque markers 730 shown in FIG. 59A, withthe resulting expected x-ray in FIG. 59B demonstrates how marker overlayis affected due to parallax. In some embodiments, an algorithm can beimplemented to account for this parallax effect. By doing so, thegraphical image indicating the position of the probe or robot 15 can beadjusted to more accurately account for the perceived shift caused byparallax.

In some embodiments, the algorithm requires information to be gatheredon the near and far plane positions of the markers 730 on the image.That is, the user can indicate, using software or an automatic scan ofthe image, the spacing between markers 730, as shown in FIG. 60, whichshows a representation of the rendering of a computer screen with anx-ray image that is affected by parallax overlaid by graphical markers732, 734 over the radio-opaque markers 730 on two plates that have thesame geometry in accordance with one embodiment of the invention. Insome embodiments, the spacing between near plane and far plane markers730 is known because of earlier calibration of the plates 700 in whichthe markers 730 are embedded, and the horizontal and vertical positionsof the markers 730 are detectable relative to the center of the image.Therefore, in some embodiments, the parallax shift of the markers 730can be calculated and applied to the mapping of any calculated threedimensional points on to the two dimensional image, and application ofany necessary positional shift. For example, in some embodiments, itmight be of interest to display a line segment on the two dimensionalimage representing how the shaft of a probe or robot 15 guide tube 50(that is being tracked using optical tracking) would appear followingx-ray imaging. In some embodiments, the x-axis, y-axis, and z-axislocation of each end of the line segment (which has been calculated fromoptical tracking data) can be shifted based on the known parallax.Further, in some embodiments, a new line segment can be displayed thatbetter represents how this projected object should appear on the 2Dx-ray image.

In some embodiments, a method of implementing this system of twoorthogonal fluoroscopy images to control a robot 15 can involvecombining a robot 15 and fluoroscopy unit into a single interconnecteddevice. There could be some advantages of this combination. For example,a conventional rotating turntable mechanism could be incorporated thatcould swing the fluoro arm into place, while at the same time swingingthe robot arm 23 out of place (since the robot 15 would typically not bein the surgical field 17 at the same time as the fluoro arm).Furthermore, in some embodiments, the size of the robot arm 23 could bereduced compared to the stand-alone robot 15 because the fluoro arm'smass would serve as a counter-balance weight to help stabilize the robotarm 23. Moreover, in some embodiments, with integration, the fluoroscopyunit can more quickly transfer the image to the computer 100 and overlaywith a graphical plot, for instance, as line segments starting at thecenter of the image and extending radially (similar to pie slices)around the image to facilitate appropriate marker 730 overlay. In someembodiments, overlaid near and far plane markers 730 should always fallon the same ray if the plates 690 with embedded markers 730 on thesubject are aligned substantially parallel (see for example FIG. 61which shows a graphical overlay for the x-ray image screen intended tohelp the user physically line up the x-ray machine). In someembodiments, the graphical overlay for the x-ray image screen can helpthe user physically line up the x-ray machine to avoid parallax. Withparallax, any pair of corresponding markers on the 2 plates should lieon the same radial line, although the one in the far plane will liecloser to the middle of the image. In some embodiments, this overlaycould be a physical object such as transparent film, or acomputer-generated graphical image. In some embodiments, lines arespaced radially by 10 degrees, but actual spacing (frequency of lines)and regions in which lines are drawn could be user selectable.

Some embodiments can include mapping a 3D anatomical coordinate systemon to two 2D orthogonal views (and vice versa) while consideringparallax. For example, in some embodiments, a rigid frame is mounted tothe patient and two perpendicular x-rays are taken to create a 3Dcoordinate system. To define this 3D coordinate system, a method isneeded to map points from the 2D views (each with parallax) to the 3Dvolume and vice versa. The 3D coordinate system has coordinates x, y, zwhile the two 2D coordinate systems have coordinates x_(AP), z_(AP) andx_(Lat), z_(Lat) (“AP” for “anteroposterior” and “Lat” for “lateral”views).

In some embodiments, it can be assumed that the x-ray path from emitterto receiver is conical, and therefore linear interpolation/extrapolationcan be used to adjust the positions of represented points. In someembodiments, software can calculate the distance of each landmark fromthe center of the image (indicated by dashed or dotted arrows). Thesedistances, together with the known distance between near plane and farplane plates, can provide the necessary information to account for theparallax shift when mapping graphical objects whose positions are knownin 3D back on to this 2D image.

Some embodiments can include solving to map x, y, z onto x_(AP), z_(AP)and x_(Lat), z_(Lat). For example, consider two intermediate 2D AP andlateral views represented as follows:

x _(ta)=(x−x _(oa))s _(AP)

z _(ta)=(z−z _(oa))s _(AP)

y _(tl)=(y−y _(ol))s _(Lat)

z _(tl)=(z−z _(ol))s _(Lat)

Where x_(ta) and z_(ta) can be called temporary scaled values of x and zin the AP plane, y_(tl) and z_(tl) are temporary scaled values of y andz in the Lat plane, s_(AP) is the scaling factor in the AP plane,determined from the known near plane¹ marker spacing. s_(Lat) is thescaling factor in the Lat plane, determined from the known near planemarker spacing of the lateral markers, and x_(oa), z_(oa), y_(ol), andz_(ol) are offsets in AP and Lat planes that position the markers suchthat they are as they appear centered about the image determined fromregistered positions of the markers on the images. In other words,(x_(ta), z_(ta))=(0, 0) represents the center of the AP image and(y_(tl), z_(tl))=(0, 0) represents the center of the lateral image.These planar values would be enough to display a 2D representation if noparallax were present or near plane markers were only being displayed.

In some embodiments, to find x_(oa), z_(oa), y_(ol), and z_(ol) considerpairs of points on the x-rays, because the ratio of distance from centeron the x-ray is the same as the ratio of distance from center on thetemporary scaled values. For example:

$\frac{x_{{AP}\; 1}}{x_{{AP}\; 2}} = {\frac{x_{{ta}\; 1}}{x_{{ta}\; 2}} = \frac{\left( {x_{1} - x_{oa}} \right)s_{AP}}{\left( {x_{2} - x_{oa}} \right)s_{AP}}}$${\left( \frac{x_{{AP}\; 1}}{x_{{AP}\; 2}} \right)\left( {x_{2} - x_{oa}} \right)} = {x_{1} - x_{oa}}$${{x_{2}\left( \frac{x_{{AP}\; 1}}{x_{{AP}\; 2}} \right)} - x_{1}} = {{x_{oa}\left( \frac{x_{{AP}\; 1}}{x_{{AP}\; 2}} \right)} - x_{oa}}$${x_{oa}\left( {\frac{x_{{AP}\; 1}}{x_{{AP}\; 2}} - 1} \right)} = {{x_{2}\left( \frac{x_{{AP}\; 1}}{x_{{AP}\; 2}} \right)} - x_{1}}$$x_{oa} = \frac{{x_{2}\left( \frac{x_{{AP}\; 1}}{x_{{AP}\; 2}} \right)} - x_{1}}{\frac{x_{{AP}\; 1}}{x_{{AP}\; 2}} - 1}$

In some embodiments, it can be seen from this equation that it isimportant to stay away from points where x_(AP1)≈x_(AP2) because itwould result in a divide by zero error. Similar equations can be writtenfor z_(oa), y_(ol), and z_(ol) as follows:

$z_{oa} = \frac{{z_{2}\left( \frac{z_{{AP}\; 1}}{z_{{AP}\; 2}} \right)} - z_{1}}{\frac{z_{{AP}\; 1}}{z_{{AP}\; 2}} - 1}$$y_{ol} = \frac{{y_{2}\left( \frac{{y_{{Lat}1}}_{\mspace{11mu}}}{y_{{Lat}2}} \right)} - y_{1}}{\frac{y_{{Lat}1}}{y_{{Lat}2}} - 1}$$z_{ol} = \frac{{z_{2}\left( \frac{{z_{{Lat}1}}_{\;}}{z_{{Lat}2}} \right)} - z_{1}}{\frac{z_{{Lat}1}}{z_{{Lat}2}} - 1}$

This mapping to temporary scaled values gets the near plane markersmapped correctly, but adjustment is needed to account for any positionother than near plane as follows:

x _(AP) =x _(ta) k _(a)(y)

z _(AP) =z _(ta) k _(a)(y)

y _(Lat) =y _(tl) k _(l)(x)

z _(Lat) =z _(tl) k _(l)(x)

As specified, k_(a) is a function of y and k_(l) is a function of x. Fork_(a), this function is a linear interpolation function, in which if yis the y position of the near plane (y_(n)), then k_(a)=1 and if y isthe y position of the far plane (y_(f)), then k_(a) is the ratio of farplane spacing to near plane spacing, r_(a). For k_(l), this function isa linear interpolation function, in which if x is the x position of thenear plane (x_(n)), then k_(l)=1 and if x is the x position of the farplane (x_(f)), then k_(l) is the ratio of far plane spacing to nearplane spacing, r_(l). Note that y_(n), y_(f), x_(n), and x_(f) are in acoordinate system with the origin at the center of the image.

$k_{a} = {1 - {\left( \frac{y_{tl} - y_{n}}{y_{f} - y_{n}} \right)\left( {1 - r_{a}} \right)}}$$k_{l} = {1 - {\left( \frac{x_{ta} - x_{n}}{x_{f} - x_{n}} \right)\left( {1 - r_{l}} \right)}}$

Combining equations,

$x_{AP} = {x_{ta}\left\lbrack {1 - {\left( \frac{y_{tl} - y_{n}}{y_{f} - y_{n}} \right)\left( {1 - r_{a}} \right)}} \right\rbrack}$$z_{AP} = {z_{ta}\left\lbrack {1 - {\left( \frac{y_{tl} - y_{n}}{y_{f} - y_{n}} \right)\left( {1 - r_{a}} \right)}} \right\rbrack}$$y_{Lat} = {y_{tl}\left\lbrack {1 - {\left( \frac{x_{ta} - x_{n}}{x_{f} - x_{n}} \right)\left( {1 - r_{l}} \right)}} \right\rbrack}$$z_{Lat} = {z_{tl}\left\lbrack {1 - {\left( \frac{x_{ta} - x_{n}}{x_{f} - x_{n}} \right)\left( {1 - r_{l}} \right)}} \right\rbrack}$

It should also be possible to map x_(AP), z_(AP), y_(Lat), and z_(Lat)onto x, y, z. Having 4 equations and 4 unknowns:

$x_{AP} = {x_{ta}\left\lbrack {1 - {\left( \frac{y_{tl} - y_{n}}{y_{f} - y_{n}} \right)\left( {1 - r_{a}} \right)}} \right\rbrack}$${1 - \frac{x_{AP}}{x_{ta}}} = {\left( \frac{y_{tl} - y_{n}}{y_{f} - y_{n}} \right)\left( {1 - r_{a}} \right)}$${\left( {1 - \frac{x_{AP}}{x_{ta}}} \right)\left( \frac{y_{f} - y_{n}}{1 - r_{a}} \right)} = {y_{tl} - y_{n}}$$y_{tl} = {{\left( {1 - \frac{x_{AP}}{x_{ta}}} \right)\left( \frac{y_{f} - y_{n}}{1 - r_{a}} \right)} + y_{n}}$

Then substitute into this equation:

$y_{Lat} = {y_{tl}\left\lbrack {1 - {\left( \frac{x_{ta} - x_{n}}{x_{f} - x_{n}} \right)\left( {1 - r_{l}} \right)}} \right\rbrack}$$y_{Lat} = {\left\lbrack {{\left( {1 - \frac{x_{AP}}{x_{ta}}} \right)\left( \frac{y_{f} - y_{n}}{1 - r_{a}} \right)} + y_{n}} \right\rbrack \left\lbrack {1 - {\left( \frac{x_{ta} - x_{n}}{x_{f} - x_{n}} \right)\left( {1 - r_{l}} \right)}} \right\rbrack}$

And solve for x_(ta):

$\mspace{20mu} {y_{Lat} = {\left\lbrack {\left( \frac{y_{f} - y_{n}}{1 - r_{a}} \right) - {\frac{x_{AP}}{x_{ta}}\left( \frac{y_{f} - y_{n}}{1 - r_{a}} \right)} + y_{n}} \right\rbrack \left\lbrack {1 - {\left( \frac{1 - r_{l}}{x_{f} - x_{n}} \right)\left( {x_{ta} - x_{n}} \right)}} \right\rbrack}}$$y_{Lat} = {\left\lbrack {\left\lbrack {\left( \frac{y_{f} - y_{n}}{1 - r_{a}} \right) + y_{n}} \right\rbrack - \left\lbrack {\frac{x_{AP}}{x_{ta}}\left( \frac{y_{f} - y_{n}}{1 - r_{a}} \right)} \right\rbrack} \right\rbrack {\quad{{\left\lbrack {\left\lbrack {1 + {x_{n}\left( \frac{1 - r_{l}}{x_{f} - x_{n}} \right)}} \right\rbrack - \left\lbrack {x_{ta}\left( \frac{1 - r_{l}}{x_{f} - x_{n}} \right)} \right\rbrack} \right\rbrack \mspace{20mu} y_{Lat}} = {{{\left\lbrack {A - \frac{B}{x_{ta}}} \right\rbrack \left\lbrack {C - {Dx}_{ta}} \right\rbrack}\mspace{20mu} y_{Lat}} = {{{AC} - \frac{BC}{x_{ta}} - {ADx}_{ta} + {{BD}\mspace{20mu} {AC}} + {BD} - y_{Lat}} = {{\frac{BC}{x_{ta}} + {{{ADx}_{ta}\mspace{20mu}({AD})}x_{ta}^{2}} + {\left( {y_{Lat} - {AC} - {BD}} \right)x_{ta}} + ({BC})} = 0}}}}}}$

Quadratic Formula:

$x = \frac{{- b} \pm \sqrt{b^{2} - {4\; {ac}}}}{2\; a}$$x_{ta} = \frac{{- \left( {y_{Lat} - {AC} - {BD}} \right)} \pm \sqrt{\left( {y_{Lat} - {AC} - {BD}} \right)^{2} - {4({AD})({BC})}}}{2({AD})}$

Where:

$A = {\left( \frac{y_{f} - y_{n}}{1 - r_{a}} \right) + y_{n}}$$B = {x_{AP}\left( \frac{y_{f} - y_{n}}{1 - r_{a}} \right)}$$C = {1 + {\left( \frac{1 - r_{l}}{x_{f} - x_{n}} \right)x_{n}}}$$D = \frac{1 - r_{l}}{x_{f} - x_{n}}$

Then plug into this equation to solve for y_(tl):

$y_{tl} = {{\left( {1 - \frac{x_{AP}}{x_{ta}}} \right)\left( \frac{y_{f} - y_{n}}{1 - r_{a}} \right)} + y_{n}}$

Then plug into this equation to solve for z_(tl):

$z_{tl} = {z_{LAT}/\left\lbrack {1 - {\left( \frac{x_{ta} - x_{n}}{x_{f} - x_{n}} \right)\left( {1 - r_{l}} \right)}} \right\rbrack}$

Then plug into this equation to solve for z_(ta):

$z_{ta} = {z_{AP}/\left\lbrack {1 - {\left( \frac{y_{tl} - y_{n}}{y_{f} - y_{n}} \right)\left( {1 - r_{a}} \right)}} \right\rbrack}$

Solve differently to give another option for z:

$y_{Lat} = {y_{tl}\left\lbrack {1 - {\left( \frac{x_{ta} - x_{n}}{x_{f} - x_{n}} \right)\left( {1 - r_{l}} \right)}} \right\rbrack}$${\left( \frac{x_{ta} - x_{n}}{x_{f} - x_{n}} \right)\left( {1 - r_{l}} \right)} = {1 - \frac{y_{Lat}}{y_{tl}}}$${\left( {x_{ta} - x_{n}} \right)\left( \frac{1 - r_{l}}{x_{f} - x_{n}} \right)} = {1 - \frac{y_{Lat}}{y_{tl}}}$${{x_{ta}\left( \frac{1 - r_{l}}{x_{f} - x_{n}} \right)} - {x_{n}\left( \frac{1 - r_{l}}{x_{f} - x_{n}} \right)}} = {1 - \frac{y_{Lat}}{y_{tl}}}$${x_{ta}\left( \frac{1 - r_{l}}{x_{f} - x_{n}} \right)} = {1 - \frac{y_{Lat}}{y_{tl}} + {x_{n}\left( \frac{1 - r_{l}}{x_{f} - x_{n}} \right)}}$$x_{ta} = {{\left( {1 - \frac{y_{Lat}}{y_{tl}}} \right)\left( \frac{x_{f} - x_{n}}{1 - r_{l}} \right)} + x_{n}}$

Substitute into:

$x_{AP} = {x_{ta}\left\lbrack {1 - {\left( \frac{y_{tl} - y_{n}}{y_{f} - y_{n}} \right)\left( {1 - r_{a}} \right)}} \right\rbrack}$$x_{AP} = {\left\lbrack {{\left( {1 - \frac{y_{Lat}}{y_{tl}}} \right)\left( \frac{x_{f} - x_{n}}{1 - r_{l}} \right)} + x_{n}} \right\rbrack \left\lbrack {1 - {\left( \frac{y_{tl} - y_{n}}{y_{f} - y_{n}} \right)\left( {1 - r_{a}} \right)}} \right\rbrack}$

And solve for y_(tl):

$\mspace{20mu} {x_{AP} = {\left\lbrack {\left( \frac{x_{f} - x_{n}}{1 - r_{l}} \right) - {\frac{y_{Lat}}{y_{tl}}\left( \frac{x_{f} - x_{n}}{1 - r_{l}} \right)} + x_{n}} \right\rbrack \left\lbrack {1 - {\left( {y_{tl} - y_{n}} \right)\left( \frac{1 - r_{a}}{y_{f} - y_{n}} \right)}} \right\rbrack}}$$x_{AP} = {\left\lbrack {\left\lbrack {\left( \frac{x_{f} - x_{n}}{1 - r_{l}} \right) + x_{n}} \right\rbrack - \left\lbrack {\frac{y_{Lat}}{y_{tl}}\left( \frac{x_{f} - x_{n}}{1 - r_{l}} \right)} \right\rbrack} \right\rbrack {\quad{{\left\lbrack {\left\lbrack {1 + {y_{n}\left( \frac{1 - r_{a}}{y_{f} - y_{n}} \right)}} \right\rbrack - \left\lbrack {y_{tl}\left( \frac{1 - r_{a}}{y_{f} - y_{n}} \right)} \right\rbrack} \right\rbrack \mspace{20mu} x_{AP}} = {{{\left\lbrack {A - \frac{B}{y_{tl}}} \right\rbrack \left\lbrack {C - {Dy}_{tl}} \right\rbrack}\mspace{20mu} x_{AP}} = {{{AC} - \frac{BC}{y_{tl}} - {ADy}_{tl} + {{BD}\mspace{20mu} {AC}} + {BD} - x_{AP}} = {{\frac{BC}{y_{tl}} + {{{ADy}_{tl}\mspace{20mu}({AD})}y_{tl}^{2}} + {\left( {x_{AP} - {AC} - {BD}} \right)y_{tl}} + ({BC})} = 0}}}}}}$

Quadratic formula:

$x = \frac{{- b} \pm \sqrt{b^{2} - {4\; {ac}}}}{2\; a}$$y_{tl} = \frac{{- \left( {x_{AP} - {AC} - {BD}} \right)} \pm \sqrt{\left( {x_{AP} - {AC} - {BD}} \right)^{2} - {4({AD})({BC})}}}{2({AD})}$

Where:

$A = {\left( \frac{x_{f} - x_{n}}{1 - r_{l}} \right) + x_{n}}$$B = {y_{Lat}\left( \frac{x_{f} - x_{n}}{1 - r_{l}} \right)}$$C = {1 + {y_{n}\left( \frac{1 - r_{a}}{y_{f} - y_{n}} \right)}}$$D = \frac{1 - r_{a}}{y_{f} - y_{n}}$

From these equations, it is possible to go from a known x,y,z coordinateto the perceived x_(AP), z_(AP) and x_(Lat), x_(Lat) coordinates on thetwo views, or to go from known x_(AP), z_(AP) and x_(Lat), z_(Lat)coordinates on the two views to an x,y,z coordinate in the 3D coordinatesystem. It is therefore possible to plan a trajectory on the x_(AP),z_(AP) and x_(Lat), z_(Lat) views and determine what the tip and tail ofthis trajectory are, and it is also possible to display on the x_(AP),z_(AP) and x_(Lat), z_(Lat) views the current location of the robot'send effectuator.

In some embodiments, additional measurement hardware (for example,conventional ultrasound, laser, optical tracking, or a physicalextension like a tape measure) can be attached to the fluoro unit tomeasure distance to the attached plates, or other points on the anatomyto ensure that plates are parallel when fluoro images are obtained.

In some embodiments, the identity of the surgical instrument 35 can beused by the control system for the computing device 3401 or othercontroller for the surgical robot system 1. In some embodiments, thecontrol system 3401 can automatically adjust axial insertion and/orforces and applied torques depending upon the identity of the surgicalinstrument 35.

In some embodiments, when performing a typical procedure for needle7405, 7410 or probe insertion (for biopsy, facet injection, tumorablation, deep brain stimulation, etc.) a targeting fixture 690 is firstattached by the surgeon or technician to the patient 18. The targetingfixture 690 is either clamped to bone (open or percutaneously), adheredas a rigid object to the skin, or unrolled and adhered to the skin (forexample using the flexible roll shown as 705 in FIG. 21A). In someembodiments, the roll 705 could have a disposable drape incorporated. Ifa flexible roll 705 is used, reflective markers 720 will then be snappedinto place in some embodiments.

In some embodiments, once a targeting fixture 690 is attached, thepatient 18 can receive an intraoperative 3D image (Iso-C, O-Arm, orintraoperative CT) with radio-opaque markers 730 included in the fieldof view along with the region of interest. In some embodiments, for bestaccuracy and resolution, a fine-slice image is preferred (CT slicespacing=1 mm or less). The 3D scan has to include the radio-opaquemarkers 730 and the anatomy of interest; not including both woulddisallow calibration to the robot 15.

In some embodiments, the 3D image series is transferred to (or acquireddirectly to) the computer 100 of the robot 15. The 3D image has to becalibrated to the robot's position in space using the locations on the3D image of the radio-opaque markers 730 that are embedded in thetargeting fixture 690. In some embodiments, this calibration can be doneby the technician scrolling through image slices and marking them usingthe software, or by an algorithm that automatically checks each slice ofthe medical image, finds the markers 730, verifying that they are themarkers 730 of interest based on their physical spacing (the algorithmis documented herein). In some embodiments, to ensure accuracy, limitsubjectivity, and to speed up the process, image thresholding is used tohelp define the edges of the radio-opaque marker 730, and then to findthe center of the marker 730 (the program is documented herein). Someembodiments of the software can do the necessary spatial transformationsto determine the location in the room of the robot's markers relative toanatomy through standard rigid body calculations. For example, byknowing the locations of the radio-opaque markers 730 in the coordinatesystem of the medical image, and knowing the locations of the activemarkers 720 on the calibration frame 700 relative to these radio-opaquemarkers 730, and monitoring the locations of the active markers on therobot 15 and targeting fixture 690.

Some embodiments allow the surgeon to use the software to plan thetrajectories for needles/probes 7405, 7410. In some embodiments, thesoftware will allow any number of trajectories to be stored for useduring the procedure, with each trajectory accompanied by a descriptor.

In some embodiments, the robot 15 is moved next to the procedure tableand cameras 8200 for tracking robot 15 and patient 18 are activated. Thecameras 8200 and robot 15 are positioned wherever is convenient for thesurgeon to access the site of interest. The marker mounts on the robot15 have adjustable positions to allow the markers 720 to face toward thecameras 8200 in each possible configuration. In some embodiments, ascreen can be accessed to show where the robot 15 is located for thecurrent Z-frame 72 position, relative to all the trajectories that areplanned. In some embodiments, the use of this screen can confirm thatthe trajectories planned are within the range of the robot's reach. Insome embodiments, repositioning of the robot 15 is performed at thistime to a location that is within range of all trajectories. Alternatelyor additionally, in some embodiments, the surgeon can adjust the Z-frame72 position, which will affect the range of trajectories that the robot15 is capable of reaching (converging trajectories require less x-yreach the lower the robot 15 is in the z-axis 70). During this time,substantially simultaneously, a screen shows whether markers 720, 730 onthe patient 18 and robot 15 are in view of the cameras 8200.Repositioning of the cameras 8200, if necessary, is also performed atthis time for good visibility.

In some embodiments, the surgeon then selects the first plannedtrajectory and he/she (or assistant) presses “go”. The robot 15 moves inthe x-y (horizontal) plane and angulates roll 62 and pitch 60 until theend-effectuator 30 tube intersects the trajectory vector (see FIGS. 3Aand 3B). In some embodiments, during the process of driving to thislocation, a small laser light will indicate end-effectuator 30 positionby projecting a beam down the trajectory vector toward the patient 18.This laser simply snaps into the top of the end-effectuator 30 tube. Insome embodiments, when the robot's end-effectuator 30 tube coincideswith the trajectory vector to within the specified tolerance, auditoryfeedback is provided to indicate that the desired trajectory has beenachieved and is being held. Alternately or additionally, in someembodiments, light of a meaningful color is projected on the surgicalfield 17. For example, in some embodiments, movement of the patient 18or robot 15 is detected by optical markers 720 and the necessary x-axis66, y-axis 68, roll 62, and pitch 60 axes are adjusted to maintainalignment.

In some embodiments, the surgeon then drives Z-frame 72 down until thetip of the end-effectuator 30 reaches the desired distance from theprobe's or needle's target (typically the skin surface). While moving,the projected laser beam point should remain at a fixed location sincemovement is occurring along the trajectory vector. Once at the desiredZ-frame 72 location, in some embodiments, the surgeon or other user canselect an option to lock the Z-tube 50 position to remain at the fixeddistance from the skin during breathing or other movement. At thispoint, the surgeon is ready to insert the probe or needle 7405, 7410. Ifthe length of the guide tube 50 has been specified and a stop on theneedle 7405, 7410 or probe is present to limit the guide tube 50 aftersome length has been passed, the ultimate location of the tip of theprobe/needle 7405, 7410 can be calculated and displayed on the medicalimage in some embodiments. As described earlier, Additionally, in someembodiments, it is possible to incorporate a mechanism at the entry ofthe guide tube 50 that is comprised of a spring-loaded plunger 54 with athrough-hole, and measures electronically the depth of depression of theplunger 54, corresponding to the amount by which the probe or needle7405, 7410 currently protrudes from the tip of the guide tube 50.

In some embodiments, at any time during the procedure, if there is anemergency and the robot 15 is in the way of the surgeon, the “E-stop”button can be pressed on the robot 15, at which point all axes exceptthe Z-frame axis 72 become free-floating and the robot's end-effectuator30 can be manually removed from the field by pushing against theend-effectuator 30.

Some embodiments can include a bone screw or hardware procedure. Forexample, during a typical procedure for conventional screw or hardwareinsertion in the spine, the patient 18 is positioned prone (or otherposition) on the procedure table, and is supported. In some embodiments,a targeting fixture 690 is attached to the patient's spine by thesurgeon or technician. In some embodiments, the targeting fixture 690 iseither clamped to bone (open or percutaneously) or unrolled and adheredto the skin (for example using roll 705). The roll 705 could have adisposable drape incorporated. If a flexible roll 705 is used,reflective markers 720 will then be snapped into place in someembodiments.

In some embodiments, once a targeting fixture 690 is attached, thepatient 18 can undergo an intraoperative 3D image (Iso-C, O-Arm, orintraoperative CT) with radio-opaque markers 730 included in the fieldof view along with the bony region of interest. In some embodiments, forbest accuracy and resolution, a fine-slice image is preferred (where theCT slice spacing=1 mm or less). The 3D scan in some embodiments has toinclude the radio-opaque markers 730 and the bony anatomy; not includingboth would disallow calibration to the robot 15.

In some embodiments, the 3D image series is transferred to (or acquireddirectly to) the computer 100 of the robot 15, and the 3D image iscalibrated in the same way as described above for needle 7405, 7410 orprobe insertion. The surgeon then uses the software to plan thetrajectories for hardware instrumentation (e.g., pedicle screw, facetscrew). Some embodiments of the software will allow any number oftrajectories to be stored for use during the procedure, with eachtrajectory accompanied by a descriptor that may just be the level andside of the spine where screw insertion is planned.

In some embodiments, the robot 15 is moved next to the table and cameras8200 for tracking robot 15 and patient 18 are activated. The cameras8200 are positioned near the patient's head. In some embodiments, themarkers for the robot 15 are facing toward the cameras 8200, typicallyin the positive y-axis 68 direction of the robot's coordinate system. Insome embodiments, a screen can be accessed to show where the robot 15 islocated relative to all the trajectories that are planned for thecurrent Z-frame 72 position. Using this screen it can be confirmed thatthe trajectories planned are within the range of the robot's reach. Insome embodiments, repositioning of the robot 15 to a location that iswithin range of all trajectories is performed at this time. Alternatelyor additionally, in some embodiments, the surgeon can adjust the Z-frame72 position, which will affect the range of trajectories that the robot15 is capable of reaching (converging trajectories require less x-yreach the lower the robot 15 is in Z). During this time, simultaneouslyin some embodiments, a screen shows whether markers 720 on the patient18 and robot 15 are in view of the cameras 8200. Repositioning of thecameras 8200, if necessary, is also performed at this time for goodvisibility.

In some embodiments, the surgeon then selects the first plannedtrajectory and he/she (or assistant) presses “go”. The robot 15 moves inthe x-y (horizontal) plane and angulates roll 62 and pitch 60 until theend-effectuator 30 tube intersects the trajectory vector. During theprocess of driving to this location, in some embodiments, a small laserlight will indicate end-effectuator 30 position by projecting a beamdown the trajectory vector toward the patient 18. This laser simplysnaps into the top of the end-effectuator guide tube 50. When therobot's end-effectuator guide tube 50 coincides with the trajectoryvector to within the specified tolerance, auditory feedback is providedin some embodiments to indicate that the desired trajectory has beenachieved and is being held. In some embodiments, movement of the patient18 or robot 15 is detected by optical markers 720 and the necessaryx-axis 66, y-axis 68, roll 62, and pitch 60 axes are adjusted tomaintain alignment.

In some embodiments of the invention, the surgeon then drives Z-frame 72down until the tip of the end-effectuator 30 reaches a reasonablestarting distance from the site of operation, typically just proximal tothe skin surface or the first tissues encountered within the surgicalfield 17. While moving, the projected laser beam point should remain ata fixed location since movement is occurring along the trajectoryvector. Once at the desired location, the user may or may not select anoption to lock the Z-tube 50 position to remain at the fixed distancefrom the anatomy during breathing or other movement.

One problem with inserting conventional guide-wires and screws into bonethrough any amount of soft tissue is that the screw or wire maysometimes deflect, wander, or “skive” off of the bone in a trajectorythat is not desired if it does not meet the bone with a trajectoryorthogonal to the bone surface. To overcome this difficulty, someembodiments can use a specially designed and coated screw specificallyintended for percutaneous insertion. Some other embodiments can use anend-effectuator 30 tip fitted with a guide tube 50 or dilator, capableof being driven all the way down to the bone. In this instance, theguide tube 50 needs to have a sharp (beveled) leading edge 30 b, and mayneed teeth or another feature to secure it well to the bone once incontact. This beveled tube 50 (i.e. guide tube 50 that includes beveledleading edge 30 b) is driven through soft tissue and next to bonethrough one of two different methods using the surgical robot system 1as described.

In applications where conventional screws are to be driven into bone,the surgeon may want to move the end-effectuator tip 30, fitted with aguide tube 50 or a conventional dilator, all the way down to the bone.Referring to FIG. 62 showing steps 6210, 6215, 6220, 6225, 6230, 6235,6240, 6245, and either 6250, 6255, 6260, and 6260, or 6270, 6275, 6280and 6285, two embodiments address this need. In some embodiments, theuser can insert the tube 50 and force it down the axis Z-tube axis 64 byhand, or with the robot 15 until a peak in force is registered bytactile feel or by a conventional force sensor on the end-effectuator 30(signaling contact with bone). At this point, it is no longer necessaryfor the tip of the drill bit 42 to be positioned past the tip of thetube 50 (in fact be better to have it slightly retracted). As describedearlier, a drill bit 42 can include a drill stop 46, and the drill bit42 can be locked and held (see for example FIGS. 17C-17E and 17F-17J).In some embodiments, the stop 46 on the drill bit 42 can then beadjusted by pulling one of the releases 48 and slightly adjusting itsposition. Then, the tube 50 can be brought up against bone and lockedthere. Now, the stop 46 can be adjusted to show how much the drill bit42 would protrude beyond the tip. This same value can be used to offset(extrapolate) the tip of the tube 50 on the software, showing the userwhere the tip of the drill bit 42 will end up.

In some embodiments, the Z-tube axis 64 is fitted with a conventionalforce sensor with continuous force readings being displayed on thescreen (such as display means 29). In some embodiments, the Z-frame 72is then driven down into tissue while continuously adjusting the x-axis66 and y-axis 68 to keep the tube 50 aligned with the trajectory vector.In some embodiments, the steps of 6210, 6215, 6220, 6225, 6230, 6235,6240, 6245, 6250 and 6255 can be used to drive the tube 50 toward thetarget. In this instance, roll 62 and pitch 60, defining orientation,should not change while moving x-axis 66, y-axis 68, and the Z-frame 72as Z-axis 70 along this vector, while holding Z-tube 50 rigidly lockedat mid-range. For this procedure, in some embodiments, the Z-tube 50stiffness must be set very high, and may require a conventionalmechanical lock to be implemented. In some embodiments, if Z-tube 50 isnot stiff enough, a counter force from the tissues being penetrated maycause it to move back in the opposite direction of Z-frame 72, and thetube 50 will not have any net advancement. In some embodiments, based onthe surgeon's previous experience and lab testing, Z-frame 72 is drivendown until a force level from the monitored force on Z-tube 50 matchesthe force typical for collision with bone (step 6260).

In some alternative embodiments, Z-tube 50 is positioned near the top ofits range and Z-frame 72 is advanced (while adjusting x-axis 66 andy-axis 68 to stay on the trajectory vector) until the tube 50 tip isnear the outermost border of dissected tissue (i.e. skin duringpercutaneous procedures). In some embodiments, the Z-tube's motor 160 isthen deactivated to allow it to move freely while still monitoring itsposition (step 6270). In some embodiments, the surgeon then pushes theend-effectuator 30 down while x-axis 66, y-axis 68, roll 62, and pitch60 adjustments can allow the tube 50 to be aligned with the trajectoryvector (step 6275). Moreover, since the Z-tube 50 is passive, in someembodiments, the surgeon can manually force the tube 50 to advance untilhe/she experiences the tactile sense of the tube hitting bone, at whichpoint the Z-tube 50 position is locked (motor 160 activated) by thesurgeon or assistant (step 6280, 6285).

At this point, in some embodiments, the guide tube 50 is adjacent tobone and the surgeon may wish to drill into the bone with a conventionalguide-wire or drill bit, or insert a screw. For screw prep andinsertion, in some embodiments, the surgeon either uses a method thatincorporates guide-wires, or a method that does not use guide-wires.

Some embodiments include a guide-wire method. For example, in someembodiments, a guide-wire is drilled into bone through the guide tube50. After the guide-wire is in place, Z-frame 72 and tube 50 are drivenupward along the trajectory vector until outside the body. In someembodiments, the tube is then released with a quick release from therobot's end-effectuator 30 so it can be positioned at the nexttrajectory. In some embodiments, a cannulated screw, already commonlyused in spine surgery, can then be driven in place over the guide-wire.

Some embodiments include a non-guide-wire method. For example, a pilothole may or may not be drilled first. In some embodiments, a screw isthen driven into bone directly through the guide tube 50, which abutsbone. In some embodiments, the tip of the screw may have the specialnon-skiving design mentioned above.

In some embodiments, if hardware other than a screw is being inserted,the surgeon may wish to dilate soft tissue. In some embodiments, adilated path would enable larger and/or more tools and implants to beinserted. In some embodiments, dilation is performed by sliding a seriesof larger and larger diameter tubes over the initial central shaft ortube. In some embodiments, a series of dilators, specially designed tointegrate to the robot's end-effectuator 30, sequentially snap on toeach other for this purpose.

In some embodiments, after the screw or hardware has been inserted inthe first trajectory, the surgeon drives the robot 15 back up thetrajectory vector away from the patient 18. In some embodiments, afterthe end-effectuator 30 is clear of the patient 18 in the Z direction,the next trajectory is selected and the robot 15 repeats the abovesteps.

In some embodiments, at any time during the procedure, if there is anemergency and the robot 15 is in the way of the surgeon, the “E-stop”button can be pressed on the robot 15, at which point all axes exceptZ-frame 72 become free-floating, and the robot's end-effectuator 30 canbe manually removed from the field by pushing against theend-effectuator.

In some embodiments, for nerve avoidance during medical procedures, aspecial conventional dilator tube (not shown) that can be used with therobot 15. In some embodiments, the dilator tube can include multipleelectrodes at its tip that can be sequentially activated to find notonly whether a nerve is nearby, but also to find which radial directionis the nearest direction toward the nerve. Some embodiments incorporatethis guide tube 50 and can identify, warn or incorporate automaticalgorithms to steer clear of the nerve.

In some embodiments, it is known that pairs of bone screws such aspedicle screws have better resistance to screw pullout if they areoriented so that they converge toward each other. In some embodiments,for the best potential biomechanical stability, a two-screw surgicalconstruct can consist of specially designed conventional screws thatwould interconnect in the X Z plane (not shown). That is, one screw canhave a socket to accept a threaded portion of the other screw so thatthe screws interconnect at their tips. A procedure such as this requiresexceptional accuracy, otherwise the screw tips would not properlyintersect, and is therefore especially well-suited for a surgical robot15. This type of hardware is useful with certain embodiments of theinvention.

In some embodiments, instead of only straight lines, the surgeon hasseveral options for trajectory planning—straight, curved or boundary forsafe-zone surgery. For curved pathway planning, in some embodiments, thesurgeon can draw a path on the medical image that has curvature of auser-selectable radius. In some embodiments, special conventionalneedles and housings can be used to execute these curved paths. In safezone surgery (tumor or trauma), in some embodiments, the surgeon firstplans a box or sphere around the region on the medical image withinwhich the probe tip, incorporating a drill or ablation instrument, willbe allowed to reside. In some embodiments, the robot 15 is driven downalong a trajectory vector either automatically or manually as describedabove to position the tip of the probe to be in the center of the safezone. In some embodiments, the surgeon would then be able pick thetool's axis of rotation (orthogonal to the long axis) based on thedesired impact he/she would like for the purpose of preserving tissueand maximizing efficiency and effectiveness for the task at hand. Forexample, in some embodiments, an axis of rotation at the surface of theskin could be selected to minimize the amount by which the tool travelslaterally and rips the skin.

In some embodiments, the robot 15 uses optical markers for tracking.Some embodiments are able to provide accurate localization of the robot15 relative to the patient 18, and utilize the LPS because of theadvantage of not being limited to line-of-sight. Additionally, in someembodiments, probes utilizing RF emitters on the tip (capable of beingtracked by the LPS) can be used for steering flexible probes inside thebody. In some embodiments, if the LPS is not yet functional forlocalization, then localization can be performed using anelectromagnetic system such as the Aurora by Northern Digital. Aurora®is a registered trademark of Northern Digital Inc. For example, in thisinstance, an electromagnetic coil and RF emitters are both present inthe probe tip. Some embodiments can offer the option of LPS orelectromagnetic localization with steerable needles 7600. In thisembodiment of the invention, the surgeon can monitor the currentlocation on the medical image where the probe tip is currentlypositioned in real-time and activate RF electrodes to advance and steerthe probe tip in the desired direction using a joystick.

As discussed earlier, in some embodiments, the end-effectuator 30 caninclude a bayonet mount 5000 is used to removably couple the surgicalinstrument 35 to the end-effectuator 30 as shown in FIG. 48. Someembodiments can include a modification to the mount 5000 allowing theability to slide a clamping piece 6300 over the spinous process 6301without full exposure of the spinous process 6301. See example FIGS.63A-63B illustrating various embodiments of an end-effectuator 30including a modified mount 5000 with a clamping piece 6300 in accordancewith at least one embodiment of the invention. As shown, the clampingpiece 6300 comprises clamps 6302 including at least one beveled edge6310, and clamp teeth 6330.

In some embodiments, the surgeon would make a stab incision in themidline and then slide the clamps 6302 of the clamping piece 6300 downalong the sides of the spinous process 6301, pushing tissue away as thetip of the clamping piece is advanced. In some embodiments, the leadingedge of the clamping mechanism 6300 would be beveled (see the leadingedges 6305 of each clamp 6302 of the clamping mechanism 6300), and havea shape similar to a periosteal elevator. This allows the clampingmechanism 6300 to separate the muscle tissue from the bony spinousprocess 6301 as it is advanced. In some embodiments, the leading edges6305 of the clamping mechanism 6300 can be electrified to enable it tomore easily slide through muscle and connective tissues to preventexcessive bleeding.

In some embodiments, a mechanism activated from farther back on theshaft (for example a turn screw 6320, or conventional spring, etc.) canbe activated to deploy clamp teeth 6330 on the clamps 6302. The samemechanism or another mechanism would close and compress the clamps 6302together to firmly secure the clamping mechanism 6300 to the spinousprocess 6301 (see FIGS. 63B-63C). Additionally, in some embodiments, ascrew 6340 aligned with the handle 6350 could deploy to thread into thespinous process 6301 (see for example, FIG. 63C).

The embodiments as described above and shown in FIGS. 63A-63C would beespecially well suited to percutaneous pedicle screw-rod surgery becausethe hole made for mounting the clamping mechanism 6300 could also beused as the hole for inserting the conventional rod to interconnect theconventional pedicle screw heads. Further, the embodiments as describedabove and shown in FIGS. 63A-63C could also be useful for mounting amarker tree (for example marker tree 795 shown in FIG. 50A) to otherbony prominences, such as transverse processes, long bones, skull base,or others.

FIGS. 64 and 65 illustrate embodiments of clamping piece 6300 actuationon a spinous process 6301 in accordance with some embodiments of theinvention. In some embodiments, the mechanism for deploying the clampteeth 6330 could be comprised of a hollow tool tip 6360 containing teeth6330 that are to one side of the hollow cavity 6370 during insertion,but are forced toward the opposite side when the mechanism is deployed,such that the embedded teeth penetrate the bone (see the illustration ofpenetrated teeth 6330 a in FIG. 64).

FIG. 65 shows an alternative embodiment of the clamping piece 6300actuation on a spinous process 6301. As shown, the groups of teeth 6330are attached to rods 6365 that run down the hollow cavities 6360 a ofthe hollow tool tips 6360. These rods 6365 pivot farther up the handle6350 (pivot point not pictured) and the clamp teeth 6330 are forcedtogether. For example, in some embodiments, rods 6365 are driven intothe hollow cavity 6360 a of the hollow tool tip 6360 on the side awayfrom the bone, forcing the clamp teeth 6330 against and into the bone(for example, see the penetrated teeth 6330 a in FIG. 65).

As described above, the opaque markers 730 must be included in a CT scanof the anatomy. However, it is desirable to crop CT scans as close aspossible to the spine to improve resolution. In some embodiments,instead of using markers 730 near where the active markers 720 arelocated, an alternative is to have a rigid extension containing opaquemarkers 730 that are temporarily attached near the spine when the scanis taken. In some embodiments, the clamping piece 6300 can be coupledwith, or otherwise modified with a targeting fixture 690. For example,FIG. 66A illustrates a clamping piece 6300 modified with a targetingfixture 690 including a temporary marker skirt 6600 in accordance withat least one embodiment of the invention, and FIG. 66B illustrates aclamping piece 6300 modified with a targeting fixture 690 as shown inFIG. 66A with the temporary marker skirt 6600 detached in accordancewith at least one embodiment of the invention. As shown, the temporarymarker skirt 6600 includes radio-opaque markers 730 in a temporary“skirt” around the base of the clamping device 6300. The design of thetemporary marker skirt 6600 and clamping device 6300 must be such thatthe markers 730 in the skirt 6600 have known locations relative to themarkers 720 for tracking that are farther away. Once the scan is taken,the opaque markers 730 are not needed. Therefore, in some embodiments,by depressing a conventional release, the skirt 6600 can be removed soit will not be in the way of the surgeon (see for example FIG. 66B).

In some embodiments, it may also be desirable to mount the targetingfixture 690 to another piece that is already rigidly attached to thepatient 18. For example, for deep brain stimulation or other brainprocedure where the patient 18 is positioned in a Mayfield head holder,the head holder could serve as an attachment point for the targetingfixture 690. Since the head holder 6700 and skull form a rigid body, itis possible to track the head holder 6700 under the assumption that theskull moves the same amount as the head holder 6700. Further, in someembodiments of the invention, a surveillance marker (such assurveillance marker 710 as illustrated in FIG. 36) could be used. Forthis targeting fixture 690, active 720 and radio-opaque 730 markerswould be rigidly attached to the head holder 6700. The radio-opaquemarkers 730 need only be in position when the scan (CT, MM, etc.) istaken and could subsequently be removed. The active markers 720 need notbe in position when the scan is taken but could instead be snapped inplace when it is necessary to begin tracking. For example, FIG. 67 showsone possible configuration for active 720 and radio-opaque markers 730attached to a Mayfield frame 6700 in accordance with one embodiment ofthe invention. As with other targeting fixtures 690, it is required thatthree or more radio-opaque markers 730 and three or more active markers720 are attached to same rigid body.

One problem with some robotic procedures is that the guide tube 50 mustbe physically rigidly mounted to the robot's end-effectuator, andtherefore mounting one or more dilator tubes can be challenging. Toaddress this problem, in some embodiments, dilators can be placed overthe central guide-tube 50 without removing the robot end-effectuator 30.For example, some embodiments can include an end-effectuator 30 thatincludes at least one dilator tube 6800, 6810. For example, FIG. 68shows end-effectuator 30 that includes nested dilators 6805 inaccordance with at least one embodiment of the invention. As shown, anested set 6805 of two or more disposable or non-disposable dilators6800, 6810 can be mounted onto the robot's end-effectuator 30. In someembodiments, each dilator 6800, 6810 may have its own removableconventional handle that allows a surgeon or an automated mechanism toforce the dilator down into soft tissue. Some embodiments could includeadditional dilators, for example, a nested set of three dilators of 7mm, 11 mm, and 14 mm diameter (not shown) may be useful for creating aportal for minimally invasive screw insertion or application of asurgical implant. In some embodiments, each dilator 6800, 6810 can havegreater length as it is closer to the central guide tube 50, allowingthe more central tube 50 to be advanced without radially advancing thedilator tubes 6800, 6810 out further.

In some further embodiments, the system 1 can include an end-effectuator30 that is coupled with at least one cylindrical dilator tube 6900. Forexample, FIGS. 69A-69C illustrate various embodiments of anend-effectuator 30 including cylindrical dilator tubes 6900 inaccordance with at least one embodiment of the invention. As shown, insome embodiments, the cylindrical dilator tubes 6900 can be formed fromtwo-halves that snap together. In some embodiments, the cylindricaldilator tubes 6900 can be formed from two-halves that snap together, andin some embodiments, the two-halves snap together over a previousdilator tube 6900. In some embodiments, the tubes 6900 can be fashionedso that they are strong in resisting radial compression, but notnecessarily strong in resisting radial expansion (since their opposingforce will be the resisting soft tissues). In some embodiments, thetubes 6900 can also benefit from a mechanism for temporarily attaching aconventional handle at the proximal end for easy insertion then removalof the handle following insertion. Moreover, some embodiments include amechanism for grasping and extracting each tube 6900 or a cluster oftubes 6900, or for attaching one or more tubes 6900 to the central guidetube 50. As depicted in FIGS. 69B and 69C, when the robot'send-effectuator 30 is raised (following the tube 6900 insertion depictedin FIG. 69B), the tube 6900 or cluster of tubes 6900 is extracted withit, leaving behind the outermost dilator 6910 a and forming a corridorfor surgery. Further, in some embodiments, the surgeon can send therobot's end-effectuator 30 to coincide with the infinite vector definingthe desired trajectory, but above the patient 18. In some embodiments,the surgeon then sends the robot's end-effectuator 30 down this vectoruntil the tip of the central guide pin or tube 50 is ready to penetratesoft tissue. In some embodiments, a starter incision may be made to helpthe central guide tube 50 penetrate the tissue surface. In someembodiments, the surgeon continues to send the robot's end-effectuator30 down the trajectory vector, penetrating soft tissue, until the targetis reached (for example, when the tube 50 abuts bone of a targetregion). Then, in some embodiments, while the robot 15 holds the centraltube 50 steady, each sequential dilator 6900 is slid down the centraltube 50 over the previous dilator 6900. When desired dilation iscomplete, in some embodiments, the proximal end of the dilator tube 6900may be secured to the patient 18 (or external assembly), and the centraltube 50 and all but the outermost dilator tube 6910 would be removed.

Some embodiments include tubes 6900 that comprise a polymeric material.In some embodiments, the tubes 6900 can include at least one eitherradiolucent or radio-opaque material. In some embodiments, dilators 6900may be radio-opaque so that their position may be easily confirmed byx-ray. Further, in some embodiments, the outermost dilator 6910 may beradiolucent so that the position of pathology drawn out through thetube, or implants, or materials passed into the patient through thetube, may be visualized by x-ray.

As described earlier, in some embodiments, the use of conventionallinear pulse motors 160 within the surgical robot 15 can permitestablishment of a non-rigid position for the end-effectuator 30 and/orsurgical instrument 35. In some embodiments, the use of linear pulsemotors 160 instead of motors with worm gear drive enables the robot 15to quickly switch between active and passive modes.

The ability to be able to quickly switch between active and passivemodes can be important for various embodiments. For example, if there isa need to position the robot 15 in the operative field, or remove therobot 15 from the operative field. Instead of having to drive the robot15 in or out of the operative field, in some embodiments, the user cansimply deactivate the motors 160, making the robot 15 passive. The usercan then manually drag it where it is needed, and then re-activate themotors 160.

The ability to be able to quickly switch between active and passivemodes can be important for safe zone surgery. In some embodiments, theuser can outline a region with pathology (for example a tumor 7300) onthe medical images (see for example FIG. 70 showing the displayed tumor7310 on display means 29). In some embodiments, algorithms may then beimplemented where the robot 15 switches from active to passive mode whenthe boundary of the region is encountered. For example, FIG. 70 showsthe boundary region 7320 within the patient 18 displayed as region 7325on the display means. Anywhere outside the boundary 7320, the robotbecomes active and tries to force the end-effectuator 30 back toward thesafe zone (i.e. within the boundary 7320). Within the boundary 7320, therobot 15 remains passive, allowing the surgeon to move the tool (such asdrill bit 42) attached to the end-effectuator 30.

In some further embodiments, the user can place restrictions (throughsoftware) on the range of orientations allowed by the tool within thesafe zone (for example, boundary 7320, and displayed as boundary 7325 inFIG. 70). In some embodiments, the tool can only pivot about a pointalong the shaft that is exactly at the level of the skin. In thisinstance, the robot 15 freely permits the surgeon to move in and out andpivot the end-effectuator 30, but does not allow left-right orfront-back movement without pivoting. For example, in some embodiments,if the surgeon wants to reach a far left point on the tumor 7300, thesurgeon must pivot the tool about the pivot point and push it to theappropriate depth of insertion to satisfy the boundary 7320 conditionsand force the tip (for example, the tip of the drill bit 42) to thatlocation. This type of limitation can be valuable because it can preventthe surgeon from “ripping” tissue as the drill is moved around todestroy the tumor 7320. Further, it also allows the surgeon to access asafe zone farther distal while keeping clear of a critical structurefarther proximal.

Some embodiments include curved and/or sheathed needles for nonlineartrajectory to a target (for example, such as a tumor 7320 describedearlier). In some embodiments, with a curved trajectory, it is possibleto approach targets inside the body of a patient 18 that might otherwisebe impossible to reach via a straight-line trajectory. For example, FIG.71A illustrates a robot end-effectuator 30 coupled with a curved guidetube 7400 for use with a curved or straight wire or tool 7410 inaccordance with at least one embodiment of the invention. In someembodiments, by forcing a curved or straight wire or tool 7410 throughthe curved guide tube 7400, at least some curvature will be imparted tothe wire or tool 7410. In some embodiments, the curved or straight wireor tool 7410 may comprise a compliant wire capable of forming to thecurvature of the guide tube 7400. In some other embodiments, the curvedor straight wire or tool 7410 may comprise a non-compliant wire, capableof substantially retaining its shape after entering and exiting theguide tube 7400. A disadvantage of using a very compliant wire is thatthe tissues that it encounters may easily force it off the desired path.A disadvantage of using a very non-compliant wire is that it would bedifficult to achieve a useful amount of curvature. Further, forcing astraight wire of intermediate compliance through a curved guide tube7400 may produce some curvature of the wire, but less curvature thanthat of the guide tube 7400. It is possible to mathematically orexperimentally model the mechanical behavior of the wire 7410 todetermine how much curvature will be imparted. For example, by knowingthe orientation of the guide tube 7400, in some embodiments, the robotmay be used to accurately guide the curved wire 7410 to a desired targetby using computerized planning to predict where the wire 7410 would endup as it traveled through tissue. Further, in some embodiments a verynon-compliant wire or tool 7410 can be manufactured in the shape of anarc with a specific radius of curvature, and then fed through a guidetube 7400 with the same radius of curvature. By knowing the orientationof the guide tube 7400 (i.e. substantially the same as wire or tool7410), computerized planning can be used to predict where the wire ortool 7410 would end up as it traveled through tissue.

Some other embodiments may use a straight guide tube 50 with a wire ortool 7410 that may be curved or straight. For example, FIG. 71Billustrates a robot end-effectuator 30 coupled with a straight guidetube 50 for use with a curved or straight wire or tool 7405, 7410 inaccordance with at least one embodiment of the invention. Some surgicalmethods may use curved needles 7410 that are manually positioned. Ingeneral, the needles consist of a rigid, straight outer guide tubethrough which is forced an inner needle 7405 with tendency to take on acurved shape. In existing manual devices, the inner needle 7405 iscomprised of nitinol, a shape memory alloy, and is formed withsignificant curvature. This curved needle 7410 is flattened and then fedthrough the outer guide tube. When it exits the other end of the guidetube, it bends with significant force back toward its original curvedconfiguration. Such a system could be adapted for use with the robot 15if the curvature of the exiting portion of the needle per unit measureexiting is known, if the radial position of the curved needle 7410relative to the straight housing is known. In some embodiments, theradial position of the curved needle 7410 can be determined by usingmarks placed on the curved and straight portions, or through anon-circular cross-section of the straight guide tube and curved needle7410 (for example, square cross-section of each). In this instance, insome embodiments, it would then be possible to preoperatively plan thepath to the target (such as a tumor 7300) and then adjust the robot 15to guide the curved wire or tool 7410 through this path. In someembodiments, the system 1 can include the ability to electricallystimulate distally while advancing a wire (for example, such as wire7405, 7410) through soft tissue. For example, some embodiments include aguide tube 7500 capable of being coupled to the robot 15 byend-effectuator 30 that is insulated along its entire shaft but has anelectrode 7510 on or near the tip (see for example FIG. 72). In someembodiments, the use of the tube 7500 to perform electromygraphy (“EMG”)can enable the system 1 to detect whether nerves come in contact withthe guide tube 7500 as the guide tube 7500 is advanced. Some alternativeembodiments can include a conventional pin (for example, stainless steelpins such as Kirschner-wires) instead of a tube 7500, insulated alongits shaft but not at the tip. In some embodiments, the wire could beconnected to a stimulator outside the body and would have the ability tostimulate distally while advancing the pin through soft tissue. In someembodiments, stimulation would allow the ability to identify criticaltissue structures (i.e., nerves, plexus).

In some further embodiments, a portion of the leading edge of the guidetube 7500 may be insulated (i.e. comprise a substantiallynon-electrically conductive area), and a portion of the leading edge maybe uninsulated (i.e. the region is inherently electrically conductivearea). In this instance, it can be possible to determine the radialdirection of the tube 7500 that is closest to the nerve by watching theresponse as the tube 7500 is rotated. That is, as the tube 7500 isrotated, the EMG nerve detection will have the most pronounced responsewhen the uninsulated portion is nearest the nerve, and the leastpronounced response when the uninsulated portion is farthest from thenerve. In some embodiments, it would then be possible for the user tomanually steer the robot 15 to automatically steer the tube 7500 fartheraway from the nerve. In addition, this modified tube 7500 could have aconventional fan-like retractor (not shown) that can be deployed togently spread the underlying muscle fibers, thereby making an entrypoint for disk removal, or screw insertion. In some embodiments, thecombination of EMG and gentle retraction can enhance the safety andoutcomes of robotic assisted spinal surgery.

As described above, one way of taking advantage of the directionalelectromyographic response is for the user to manually rotate the tube7500. In some other embodiments, the tube 7500 can be to continuouslyoscillated back and forth, rotating about its axis while potentials aremonitored. In some embodiments, to achieve the same function withoutrotating the tube 7500, the leading edge of the tube 7500 could haveconductive sections that could be automatically sequentially activatedwhile monitoring potentials. For example, in some embodiments, an arrayof two, three, four, or more electrodes 7510 (shown in FIG. 72) can bepositioned around the circumference of the leading edge of the tube7500. As shown in FIG. 72, regions 7511 between the electrodes 7510 areinsulated from each other (because the outer surface of 7500 isinsulated). In some embodiments, the electrodes 7510 can be sequentiallyactivated at a very high rate while recording potentials, andcorrelating which electrode produces the greatest response.

Some embodiments can include a steerable needle capable of being trackedinside the body. For example, U.S. Pat. No. 8,010,181, “System utilizingradio frequency signals for tracking and improving navigation of slenderinstruments during insertion in the body”, herein incorporated byreference, describes a steerable flexible catheter with two or more RFelectrodes on the tip, which are used for steering. According to themethod described in U.S. Pat. No. 8,010,181, the side or sides of thetip where the electrodes emit RF have less friction and therefore theprobe will steer away from these sides.

In some embodiments of the invention, a steerable needle 7600 can becoupled with the system 1. In some embodiments, the system 1 can includea steerable needle 7600 coupled with the robot 15 through a coupledend-effectuator 30, the steerable needle 7600 capable of being trackedinside the body of a patient 18. For example, FIG. 73 illustrates asteerable needle 7600 in accordance with at least one embodiment of theinvention. In some embodiments, steerable needle 7600 can comprise aplurality of flattened angled bevels 7605 (i.e. facets) on the tip ofthe probe, with each flat face of each bevel 7605 having an RF electrode7610. A magnetic coil sensor 7620 embedded within the needle 7600 canenable localization of the tip adjacent to the electrodes 7610. In someembodiments, RF (using for example RF transmitters 120 describedearlier) can be used for steering, whereas localization would useelectrodes 7610 with the magnetic coil sensor 7620. Some embodiments asdescribed may use off-the-shelf electromagnetic localization system suchas the Aurora® from Northern Digital, Inc. (http://www.ndigital.com),which has miniature coils capable of fitting inside a catheter.

During surgical procedures, pedicle screws or anterior body screws areinserted in two locations. However, there is a chance of failure due toscrew pullout. To enhance resistance to pullout, screws are angledtoward each other. For example, some embodiments can includeintersecting and interlocking bone screws 7700 such as those illustratedin FIG. 74, illustrating one embodiment of intersecting and interlockingbone screws 7700 in accordance with at least one embodiment of theinvention. As shown, bone screws 7700 can be coupled and can intersectand interlock 7720. In some embodiments, the intersecting andinterlocking bone screws 7700 as shown can be removed without destroyinga large area of bone.

Some embodiments of the system 1 can include conventional trackingcameras with dual regions of focus. For example, camera units such asOptotrak® or Polaris® from Northern Digital, Inc., can be mounted in abar so that their calibration volume and area of focus are set.Optotrak® or Polaris® are registered trademarks of Northern Digital, Inc(see for example FIG. 81 showing camera bar 8200). In some embodiments,when tracking the robot 15 and targeting fixture 690 with opticaltrackers (for example, active markers 720), maintaining markers 720within the center of the volume can provide the best focus. However, itis not possible for both the targeting fixture's 690 markers and therobot's 15 markers to be substantially centered simultaneously, andtherefore both are offset from center by substantially the samedistance.

In some embodiments, one solution to this issue is to set up two pairsof cameras 8200 with one camera shared, that is, cameras 1 and 2 formone pair, and cameras 2 and 3 form another pair. This configuration isthe same as the Optotrak® system (i.e., three cameras in a single bar),however, the Optotrak® only has one volume and one common focal point.Conversely, some embodiments of the invention would be tuned to have twofocal points and two volumes that would allow both the targeting fixture690 and the robot 15 to be centered at the same time. In someembodiments, the orientations of the lateral cameras can be adjusted byknown amounts with predictable impact on the focal point and volume.

In a further embodiment of the invention, two separate camera units (forexample, two Polaris® units) can be mounted to a customized conventionalbracket fixture including adjustment features (not shown). In someembodiments, this fixture would be calibrated so that the vectorsdefining the directions of the volumes and distance to focal point canbe adjustable by known amounts. In some embodiments, the user could thenpoint one Polaris® unit at the robot's markers, and the other Polaris®unit at the targeting fixture's 690 markers 720. The position of theadjustment features on the bracket would tell the computer what thetransformation is required to go from one camera's coordinate system tothe other.

In some further embodiments, the cameras 8200 (such as Optotrak® orPolaris®) focused on a particular region could be further improved by aconventional automated mechanism to direct the cameras 8200 at thecenter of the target. Such a method would improve accuracy because ingeneral, image quality is better toward the center of focus than towardthe fringes. In some embodiments, conventional motorized turrets couldbe utilized to adjust azimuth and elevation of a conventional bracketassembly for aiming the cameras 8200 (and/or in conjunction withmovement of cameras 8200 on camera arm 8210 as shown in FIG. 81). Insome embodiments, feedback from the current location of active markers720 within the field of view would be used to adjust the azimuth andelevation until the camera 8200 points directly at the target,regardless of whether the target is the center (mean) of the markers 720on the robot 15, the center of markers 720 on the targeting fixture 720,or the center of all markers 720. In some embodiments, such a methodwould allow the center of focus of the cameras 8200 to continuously moveautomatically as the patient 18 or robot move, ensuring the optimalorientation at all times during the procedure.

Some embodiments can include a snap-in end-effectuator 30 with attachedtracking fixtures 690 (including active markers 720). For example, someembodiments include snap-in posts 7800 attached to the end-effectuator30 and tracking fixtures 690. In some embodiments, the snap-in posts7800 can facilitate orienting tracking markers 720 to face cameras 8200in different setups by allowing markers 720 to be mounted to eachend-effectuator 30. FIG. 75A-75B illustrates configurations of a robot15 for positioning alongside a bed of a patient 18 that includes atargeting fixture 690 coupled to an end-effectuator 30 using a snap-inpost 7800. In some embodiments, with the robot 15 in a typicalconfiguration alongside a bed with the patient's 18 head toward theleft, one end-effectuator 30 could have right-facing markers 720(fixture 690) (illustrated in FIG. 75A) for cameras 8200 positioned atthe foot of the bed. In some embodiments, the same type ofend-effectuator 30 could have left-facing markers 720 (fixture 690) forcameras 8200 positioned at the head of the bed (illustrated in FIG.75B). In some embodiments, the fixtures 690 are mounted where they wouldbe closer to the cameras 8200 than the end-effectuator 30 so that thesurgeon does not block obscure the markers 720 from the camera whenusing the tube 50. In some further embodiments, each interchangeableend-effectuator 30 could include conventional identificationelectronics. For example, in some embodiments, each interchangeableend-effectuator 30 could include an embedded conventional chip and apress-fit electrical connector. In some embodiments, when the system 1includes a snap-in end-effectuator 30 with attached tracking fixtures690, the computer 100 may recognize which end-effectuator is currentlyattached using the identification electronics. In some embodiments, whenthe system 1 includes a snap-in end-effectuator 30 with attachedtracking fixtures 690, the computer 100 may recognize whichend-effectuator is currently attached using the identificationelectronics, and apply stored calibration settings.

The robot system 1 contains several unique software algorithms to enableprecise movement to a target location without requiring an iterativeprocess. In some embodiments, an initial step includes a calibration ofeach coordinate axis of the end-effectuator 30. During the calibration,the robot 15 goes through a sequence of individual moves while recordingthe movement of active markers 720 that are temporarily attached to theend-effectuator (see FIG. 76). From these individual moves, which do nothave to fall in a coordinate system with orthogonal axes, the requiredcombination of necessary moves on all axes is calculated.

In some embodiments, it is possible to mount optical markers 720 fortracking the movement of the robot 15 on the base of the robot 15, thento calculate the orientation and coordinates of the guide tube 50 basedon the movement of sequential axes (see earlier description related toFIG. 16). The advantage of mounting markers 720 on the base of the robot15 is that they are out of the way and are less likely to be obscured bythe surgeon, tools, or parts of the robot. However, the farther away themarkers 720 are from the end-effectuator 30, the more the error isamplified at each joint. At the other extreme, it is possible to mountthe optical markers 720 on the end-effectuator 30 (as illustrated inFIG. 76). The advantage of mounting markers 720 on the end-effectuatoris that accuracy is maximized because the markers 720 provide feedbackon exactly where the end-effectuator 30 is currently positioned. Adisadvantage is that the surgeon, tools, or parts of the robot 15 caneasily obscure the markers 720 and then the end-effectuator's 30position in space cannot be determined.

In some embodiments, it is possible to mount markers 720 at eitherextreme or at an intermediate axis. For example, in some embodiments,the markers 720 can be mounted on the x-axis 66. Thus, when the x-axis66 moves, so do the optical markers 720. In this location, there is lesschance that the surgeon will block them from the cameras 8200 or thatthey would become an obstruction to surgery. Because of the highaccuracy in calculating the orientation and position of theend-effectuator 30 based on the encoder outputs from each axis, it ispossible to very accurately determine the position of theend-effectuator 30 knowing only the position of the markers on thex-axis 66.

Some embodiments include an algorithm for automatically detecting thecenters of the radio-opaque markers 730 on the medical image. Thisalgorithm scans the medical image in its entirety looking for regionsbounded on all sides by a border of sufficient gradient. If furthermarkers 730 are found, they are checked against the stored locations andthrown out if outside tolerance.

Some biopsy procedures can be affected by the breathing process of apatient, for example when performing a lung biopsy. In some procedures,it is difficult for the clinician to obtain a sample during the correctbreathing phase. The use of tracking markers 720 coupled to a bone ofthe patient cannot alone compensate for the breathing induced movementof the target biopsy region. Some embodiments include a method ofperforming a lung biopsy with breathing correction using the system 1.Currently, for radiation treatment of lung tumors, breathing ismonitored during CT scan acquisition using a “bellows” belt (see forexample CT scanner 8000 in FIG. 77, with bellows image 8010. The bellowsmonitors the phase of breathing, and when the clinician tells thepatient to hold their breath, CT scan of the patient 18 is performed.The bellows output 8010 shows the phase in which the CT was taken.Later, targeted radiation bursts can be applied when the lung is in theright position as monitored by the bellows during the treatment phase. ACT scan is taken while the bellows monitors the breathing phase and whenthe patient held their breath during the CT scan. Later, radiationbursts are applied instantaneously when that same phase is reachedwithout requiring the patient 18 to hold their breath again.

Some embodiments include a method of performing a lung biopsy withbreathing correction using the system 1. In some embodiments, a trackingfixture 690 is attached to the patient 18 near biopsy site and bellowsbelt on the patient's 18 waist. In some embodiments, a CT scan of thepatient 18 is performed with the patient holding their breath, and whilemonitoring the breathing phase. In some embodiments, a clinician locatesthe target (for example, a tumor) on the CT volume, and configures therobot 15 to the target using at least one of the embodiments asdescribed earlier. In some embodiments, the robot 15 calibratesaccording to at least one embodiment described earlier. In someembodiments, the robot 15 moves into position above the biopsy sitebased the location of at least one tracking marker 720, 730. In someembodiments, the bellows belt remains in place, whereas in otherembodiments, the markers 720, 730 on the patient 18 can track thebreathing phase. In some embodiments, based on the bellows or trackingmarkers 720, 730, the computer 100 of the computing device 3401 withinplatform 3400 can use robotic guidance software 3406 to send a triggerduring the calibrated breathing phase to deploy a biopsy gun to rapidlyextract a biopsy of the target (such as a tumor). In some embodiments, aconventional biopsy gun (or tool, such as biopsy gun tip 8100 in FIG.78) could be mounted in the robot's end-effectuator 30 and activated bya conventional mechanism (such as for example, by a toggled digitaloutput port). For example, as shown, the biopsy gun tip 8100 cancomprise a biopsy needle 8110 including a stroke length 8120, a samplingwindow 8130 and a biopsy tip 8140. In some embodiments, the biopsyneedle 8110 in the biopsy gun tip 8100 can be mounted to theend-effectuator 30. In some embodiments, the biopsy needle 8110 can beinserted (under guidance by the robot 15) at least partially into thesuperficial tissues near the target (for example, the moving lungtumor). In some embodiments, the biopsy gun tip 8100 can fire asdirected by a software 3406 trigger, requiring only a small penetrationto retrieve the biopsy.

Deep brain stimulation (“DBS”) requires electrodes to be placedprecisely at targets in the brain. Current technology allows CT and MRIscans to be merged for visualizing the brain anatomy relative to thebony anatomy (skull). It is therefore possible to plan trajectories forelectrodes using a 3D combined CT/MRI volume, or from CT or MRI alone.Some embodiments include robot 15 electrode placement for asleep deepbrain stimulation using the system 1 where the acquired volume can thenbe used to calibrate the robot 15 and move the robot 15 into position tohold a guide 50 for electrode implantation.

In some embodiments, a Mayfield frame 6700 modified including onepossible configuration for active and radio-opaque markers (shown inFIG. 67 in accordance with one embodiment of the invention) can be usedfor electrode placement for asleep deep brain stimulation. In someembodiments, the active markers 720 do not need to be attached at thetime of the scan as long as their eventual position is unambiguouslyfixed. In some embodiments, the radio-opaque markers 730 can be removedafter the scan as long as the relative position of the active markers720 remains unchanged from the time of the scan. In some embodiments,the marker 730 can be a ceramic or metallic sphere, and for MRI, asuitable marker is a spherical vitamin E capsule. In some embodiments,the end-effectuator 30 can include an interface for feeding in aconventional electrode cannula and securing the electrode housing to theskull of the patient 18 (for example, using a Medtronic StimLoc® leadanchoring device to the skull). StimLoc® is a trademark of Medtronic,Inc., and its affiliated companies.

In some embodiments, the system 1 can perform the method steps 7910-7990as outlined in FIG. 79 for DBS electrode placement. As show, in someembodiments, the patient 18 can receive an MRI 7910, and the target andtrajectory can be planned 7915. Surgery can be initiated under generalanesthesia 7920, and the head frame (as shown in FIG. 67) can beattached to the patient 18 with three screws in the skull 7925. In someembodiments, a CT scan can be performed 7930, and the previouslyobtained MRI 7910 can be merged with the CT scan 7935. During the CTscan, software can automatically register the anatomy relative to themarkers 720, 730 that are mounted on the head holder. In someembodiments, the robot 15 can direct a laser at the skin of the patient18 to mark flaps 7940. In some embodiments, the skin of the patient 18can be prepared and draped 7945, and scalp flaps can be prepared 7950.As shown, in some embodiments, the robot 15 can laser drill entry holes7955, and the StimLoc can be secured bilaterally 7960 (permanentimplant, 2 screws per electrode). In some embodiments, the robot 15 canauto-position a conventional electrode guide adjacent to entry point ata fixed (known) distance from target 7965. In some embodiments, the duracan be opened, a cannula and electrode inserted, and a StimLoc clip canbe positioned 7970. In some embodiments, steps 7965, 7970 are repeatedfor the other side of the patient's skull 7975. In some embodiments, averification CT scan is performed 7980, a cap is placed over theStimLoc, and the flaps are closed.

In some embodiments, the robot system 1 includes at least one mountedcamera. For example, FIG. 81 illustrates a perspective view of a robotsystem including a camera arm in accordance with one embodiment of theinvention. In some embodiments, to overcome issues with line of sight,it is possible to mount cameras for tracking the patient 18 and robot 15on an arm 8210 extending from the robot. As shown in FIG. 81, in someembodiments, the arm 8210 is coupled to a camera arm 8200 via a joint8210 a, and the arm 8210 is coupled to the system 1 via joint 8210 b. Insome embodiments, the camera arm 8200 can be positioned above a patient(for example, above a patient 18 lying on a bed or stretcher as shown inFIG. 81). In this position, in some embodiments, it might be less likelyfor the surgeon to block the camera when the system 1 is in use (forexample, during a surgery and/or patient examination). Further, in someembodiments, the joints 8210 a, 8210 b can be used to sense the currentposition of the cameras (i.e. the position of the camera arm 8200).Moreover, in some embodiments, the exact position of the end-effectuator30 in the camera's coordinate system can be calculated based onmonitored counts on each robot axis 66, 68, 70, 64, and in someembodiments, the cameras 8200 would therefore only have to track markers720 on the patient 18.

Some embodiments include an arm 8210 and camera arm 8200 that can foldinto a compact configuration for transportation of the robot system 1.For example, FIG. 82A illustrates a front-side perspective view of arobot system including a camera arm in a stored position, and FIG. 82Billustrates a rear-side perspective view of a robot system including acamera arm in a stored position in accordance with one embodiment of theinvention.

Some embodiments can include methods for prostate 8330 immobilizationwith tracking for imaged-guided therapy. In some embodiments, to enablethe insertion of a needle (7405, 7410, 7600, 8110 for example) into theprostate 8330 utilizing 3D image guidance, a 3D scan of the prostate8330 relative to reference markers 720, 730 or other tracking system3417 is needed. However, the prostate 8330 is relatively mobile and canshift with movement of the patient 18. In some embodiments, it may bepossible to immobilize the prostate 8330 while also positioning andsecuring tracking markers 720 in close proximity to improve tracking andimage guidance in the prostate 8330.

The prostate 8330 is anatomically positioned adjacent to the bladder8320, the pubic bone 8310, and the rectum 8340 (see for example FIG. 83showing a lateral illustration of a patient lying supine, depicting thenormal relative positions of the prostate 8330, rectum 8340, bladder8320, and pubic bone 8310). This position facilitates entrapment of theprostate 8330, especially when it is enlarged, against the bladder 8320and pubic bone 8310 via anterior displacement applied within the rectum8340. In some embodiments, displacement could be applied using a balloon8410, a paddle 8420, or a combination of the two elements. For example,FIG. 84A shows a lateral illustration of a patient lying supine, showinghow inflation of a balloon can cause anterior displacement of theprostate 8330 toward the pubic bone 8310, and a controllable amount ofcompression against the pubic bone 8310 in accordance with oneembodiment of the invention. Further, FIG. 84B shows a lateralillustration of a patient lying supine, showing how shifting of a paddlein the rectum 8340 can cause anterior displacement of the prostate 8330toward the pubic bone 8310, and a controllable amount of compressionagainst the pubic bone 8310 in accordance with one embodiment of theinvention.

In some embodiments, the balloon 8410 has the advantage that it can beinserted into the rectum 8340 un-inflated, and then when inflated. Insome embodiments, it will displace the wall of the rectum 8340 andprostate 8330 laterally toward the pubic bone 8310. In some embodiments,a paddle 8420 can cause lateral displacement of the rectal wall andprostate 8330 if a pivot point near the anus is used.

In some embodiments, it is possible to configure a device consisting ofa balloon 8410 and paddle 8420 such that fiducials are embedded in thedevice, with these fiducials being detectable on the 3D medical image(for instance, such as MM). For example, FIG. 85 shows a sketch of atargeting fixture and immobilization device to be used for tracking theprostate 8330 during image-guided surgical procedures in accordance withone embodiment of the invention. As shown, active tracking markers 720can be rigidly interconnected to the paddle element 8420 such that thesetracking markers 720 protrude from the rectum 8340 and are visible totracking cameras (for example, 8200) during the medical procedure. Forexample, FIG. 86 shows an illustration of the device as illustrated inFIG. 85, in place in the rectum 8340 with prostate 8330 compressed andimmobilized and tracking markers visible protruding caudal to the rectum8340 in accordance with one embodiment of the invention.

In some embodiments, in addition to applying lateral force from the sideof the rectum 8340, it is also possible to apply lateral force from theside of the abdomen of the patient 18. In some embodiments, thissecondary lateral force, used in conjunction with the force from therectal wall, may assist in keeping the prostate 8330 immobilized.Additionally, it can serve as a support to which the tracking markers720 are attached, and can serve as a support to which the rectalpaddle/balloon 8420, 8410 can be attached for better stabilization. Insome embodiments, the abdominal support can consist of a piece thatpresses from anterior toward posterior/inferior to press against the topof the bladder 8320 region. For example, conventional straps or piecesthat encircle the legs can provide additional support. Since theabdominal shape and leg shape varies among patients, some customizationwould be beneficial. In some embodiments, adjustable straps and supportsmade of thermoplastic material could be utilized for customization. Insome embodiments, commercially available thermoplastic supports (forexample, from Aquaplast Inc) can be used. In some embodiments, thesupports are formed by first dipping the support material in hot waterto soften it, then applying the support to the patient's skin andmolding it. After removing the support material from the hot water, thetemperature is low enough that it does not burn the skin, but is warmenough that the support material remains soft for 1-5 minutes. In someembodiments, when the support cools, it maintains the skin contoursagainst which it has been formed. In some embodiments, this type ofsupport could be made for immobilizing the prostate 8330 shaped likemoldable briefs. In this instance, the support would be dipped in hotwater and then external straps and/or manual pressure would be appliedto force the support device to press down toward the prostate 8330.Further, in some embodiments, the support could be manufactured in twohalves, formed so that it is molded while two halves are tied together,and then removed (untied) when cool (so that it can later be reattachedin the same configuration during the procedure).

In some embodiments, the combination of the elements as described above(including balloon 8410 and/or paddle 8420, enables real-time trackingof the prostate 8330, and manual or robotically assisted insertion ofneedles (for example, 7405, 7410, 7600, 8110) into the prostate 8330based on targeting under image guidance. In some embodiments, theprocedure can include the conventional abdominal support device asdescribed above. The device would be prepared by dipping in hot wateruntil soft, then applying to the patient such that gentle pressure ismaintained from anterior to posterior/inferior against the bladder 8320region and prostate 8330. In some embodiments, under palpation, thetracking device (paddle 8420 with coupled fixture 690 including markers720 illustrated in FIG. 85) would be inserted into the rectum 8340 withthe paddle 8420 and radio-opaque markers 730 adjacent to the prostate8330. In this instance, gentle pressure can be manually applied to theprotruding handle by the surgeon to maintain the position of theinterior paddle 8420. In some embodiments, the balloon 8410 is inflatedto maintain gentle compression against the prostate 8330, and toimmobilize the prostate 8330 against the pubic bone 8310. In someembodiments, if the conventional abdominal device is used, the abdominaldevice is interconnected to the rectal device at this point foradditional stability. In some embodiments, an MM is obtained. During theMM, the active tracking markers 720 are not attached since they aremetallic. In some embodiments, sockets or other conventionalquick-connect mechanical device are present in the locations where themarkers 720 or marker tree (fixture 690) will later be inserted. In someembodiments, the MM captures an image of the prostate 8330, and theradio-opaque markers 730 embedded in the handle 8425. In someembodiments, the MM can be captured with the patient's legs down toallow the patient 18 to fit into the gantry of the scanner. In someembodiments, the patient 18 is positioned on the procedure table withlegs raised. Tracking markers 730 are snapped into the sockets on theprotruding handle or the marker tree 690 with markers 720 is otherwisefastened. In some embodiments, registration of the markers 730, 720 isachieved by software (for example, using one or more modules of thesoftware 3406 using the computing device 3401), which automaticallydetects the positions of the radio-opaque markers 730 on the medicalimage. In some embodiments, the known relative positions of the activetracking markers 720 and the radio-opaque marker 730 fiducialssynchronizes the coordinate systems of the anatomy of the patient 18,tracking system 3417 and software 3406, and robot 15. In someembodiments, the surgeon plans trajectories for needle 7405 insertioninto the prostate 8330 on the medical image, and the robot 15 moves theguide tube 50 to the desired 3D location for a needle 7405 of knownlength to be inserted to the desired depth.

Some embodiments can use a dual mode prostate 8330 tracking forimage-guided therapy. For example, in some embodiments, it is possibleto accurately track the prostate 8330 using a combination of twotracking modalities, including fiber optic tracking. For this alternatemethod to be used, an optical tracker (fiber optic probe 8700) wouldfirst be applied externally. This probe 8700 would be registered to the3D medical image (for example, using an MM scan) in substantially thesame way as previously described, such as for the spine tracking usingCT imaging. In some embodiments, after registering and calibrating sothat the coordinate systems of the medical image and cameras 8200 aresynchronized, a means of updating and correcting for movement of theprostate 8330 can be used. In some embodiments, the probe 8700 cancomprise a fiber optic sensor with a Bragg grating. For example, FIG.87. illustrates a demonstration of a fibre Bragg grating (“FBG”)interrogation technology with a flexible fiber optic cable in accordancewith one embodiment of the invention. As shown the technology isavailable from Technobis Fibre Technologies, Uitgeest, Holland. As thefiber optic cable is bent by hand, the system accurately senses theposition to which the cable deforms. As depicted in FIG. 88, in someembodiments, the probe 8700 could be inserted into the urethra with thetip of the sensor positioned at the prostate 8330. Since the prostate8330 surrounds the urethra, a sensor such as probe 8700 positioned inthe urethra should show very accurately how the prostate 8330 moves. Asshown, the probe 8700 can be coupled with the fixture 690 includingmarkers 720, 730, and coupled to the computer 100 with optical trackerelectronics 8810 and fiber optic electronics 800 coupled to the computer100, coupled to the robot 15.

In some embodiments, markings 8910 (gradations) capable of beingvisualized on MRI can be placed on the outer shaft of the probe 8700(see for example, FIG. 89). In some embodiments, if the MRI is obtainedwhile the probe 8700 is in position in the urethra, it is possible todetermine which point or points along the length of the probe 8700represent key landmarks within the prostate 8330 (e.g., distal entry,proximal exit, midpoint). In some embodiments, these points can then betracked by the fiber optic electronics 8800 during the procedure. Insome embodiments, the points can then be used to adjust the coordinatesystem of the prostate 8330 so that the local coordinate system remainsproperly synchronized with the coordinate system of the optical trackingsystem 3417 even if the surrounding anatomy (specifically the anatomy towhich the tracking markers 720, 730 are attached) shifts relative to theprostate 8330. In other words, the position of the tracking markers 720,740 on the patient's skin surface gives an approximate estimate of wherethe prostate 8330 is currently located, and the fiber optic probe 8700(which is rigidly interconnected to the tracking fixture 690) correctsthis position to substantially improve accuracy and account for shiftingof the prostate 8330.

In some embodiments, image-guided therapy can be performed using one ormore of the embodiments as described. For example, in some embodiments,the fiber optic probe as depicted in FIG. 88 can include opticallyvisible 8900 and MM visible 8910. In some embodiments, the probe 8700 isinserted into the penis and advanced until the tip passes into thebladder 8320 (shown in FIG. 89). In some embodiments, the marking 8900,8910 will provide information about what section of the fiber optic ispositioned within the prostate 8330. In some embodiments, the depth ofinsertion is recorded based on visible markings 8900 on the proximal endthat has not entered the penis is recorded. In some embodiments, thisinformation can be used to check whether the probe 8700 has moved, or toreposition the probe 8700 if it is intentionally moved. In someembodiments, the proximal end may be secured (taped) to the penis toprevent advancement or withdrawal with patient 18 movement. In someembodiments, the distal end may have a feature to prevent it from easilysliding back out of the bladder 8320. For example, as shown in FIG. 90,some embodiments include a probe 8700 that comprises an inflatable tip8920. In some embodiments, the inflatable tip 8920 can be enlarged orflared in the area near the tip. In some embodiments the tip 8920comprises a balloon that is inflatable after the tip has passed into thebladder 8320, whereas in other embodiments, the tip 8920 comprisesconventional soft wings that deploy after the tip has passed into thebladder 8320. As shown in FIG. 90, in some embodiments, a targetingfixture 690 is attached to the patient 18 in the region of the perineum(or abdomen or other suitable surface). The targeting fixture hasembedded radio-opaque fiducial markers 730 that will show up on the MM(or other 3D scan), and is equipped with a conventional quick-connectinterface that will later accept an attachment with active trackingmarkers 720. These tracking markers 720 do not need to be present yet,especially if they are not MRI compatible. The targeting fixture 690 canbe rigidly interconnected with the proximal end of the fiber optic probe8700.

In some embodiments, the patient 18 is positioned outside or in thegantry of the MRI scanner before scanning. In some embodiments, thefiber optic tracking system 9100 is briefly activated to record positionof the fiber optic probe 8700 along its entire length for laterreference (see FIG. 91). Once recorded, the electronic interface (8800)for the fiber optic tracking system 9100 may be disconnected and removedfrom the MRI area.

In some embodiments, an MRI scan is obtained. The scan must visualizethe prostate 8330, the radio-opaque fiducials 730 on the targetingfixture 690, and the markings 8910 that are present along the urethraltube that will be tracked with fiber optic probe 8700. In someembodiments, the position of the prostate 8330 along the fiber opticprobe 8700 at the time of the scan is recorded from the radio-opaquemarkings 8910 on its surface.

In some embodiments, the patient is positioned on the procedure table,and optical tracking markers 720 are snapped into the targeting fixture(see FIG. 92) and activated. In some embodiments, registration of themarkers 720 is achieved by software (for example by one or more moduleswithin the device 3401), which automatically detects the positions ofthe radio-opaque markers 730 on the medical image. The known relativepositions of the active tracking markers 720 and the radio-opaquefiducials 730 synchronizes the coordinate systems of the anatomy,tracking system 3417, and robot 15. The fiber optic tracking system 9100is activated.

In some embodiments, the offset of the prostate 8330 from the positionrecorded on the MRI scan is determined as the offset of the prostate8330 in the optically sensed position of the probe 8700 relative to theposition at the time of the MM scan. In some embodiments, the surgeonplans trajectories for insertion of the needle 7405 into the prostate8330 (from the medical image), and the robot 15 moves the guide tube 50to the desired 3D location for a needle 7405 to be inserted to thedesired depth (see FIG. 93). In some embodiments, an offset necessary toensure that the correct region of the prostate 8330 is targeted, isdetermined from the probe 8700 sensed offset, and the position of theguide tube 50.

In some other embodiments, the probe 8700 could be inserted down theesophagus to track movement of the stomach, intestines, or any portionof the digestive system. In some embodiments, it could be inserted intoa blood vessel to track the position of major vessels inside the body.In some embodiments, it could be inserted through the urethra into thebladder, ureters, or kidney. In all cases, it would help localizeinternal points for better targeting for therapy.

In some further embodiments, the probe 8700 could be combined with aconventional catheter for other uses. For example, fluid could beinjected or withdrawn through a hollow conventional catheter that isattached along its length to the probe 8700. Further, in someembodiments, a conventional balloon catheter could also be utilized. Theballoon could be temporarily inflated to secure a portion of the probe8700 within the urethra, or other position inside the body, ensuringthat the probe 8700 does not move forward or backward once positionedwhere desired.

A number of technologies for real-time 3D visualization of deformingsoft tissue and bony anatomy without the radiation are available and/orare in development. In some embodiments, the surgical robot 15 can usethese technologies during surgery, or other image-guided therapy. Insome embodiments, the use of real-time 3D visualization, automatednon-linear path planning and automated steering and advancement offlexible catheters or wires (for example wires 7405, 7410, 8600, or8110) in a non-linear path becomes increasingly important.

In some embodiments, it may be possible to visualize soft tissues inreal time by combining MRI (magnetic resonance imaging) and ultrasoundor contrast enhanced ultrasound (“CEUS”). For example, in someembodiments, an MRI scan and a baseline ultrasound scan would beobtained of the anatomy of interest. In some embodiments, landmarksvisualized on the ultrasound would be correlated to the MM (for example,borders of organs, blood vessels, bone, etc.). In some embodiments, adiscrete set of key landmarks could be correlated such that the movementof other points of interest between these landmarks could beinterpolated. In some embodiments, a computerized geometric model (withits unmoved baseline position corresponding to the anatomy seen on theMRI) would be created. Then, when movements of the landmark points aredetected on ultrasound, the positions of the corresponding tissuesvisualized on the model can be adjusted. In some embodiments, theultrasound would be allowed to run continuously, providing real-timedata on the positions of the landmarks. In some embodiments, changes inlandmark position would be used to update the model in real time,providing an accurate 3D representation of the soft tissues withoutexposure to radiation. In some embodiments, optical tracking markers 720attached to the conventional ultrasound probes could provide data on themovement of the probes relative to the anatomy, which would affect themodel calibration. In some embodiments, for accurate 3D positions of thepoints on the soft tissues, it may be necessary to utilize severalconventional ultrasound probes locked in a rigid orientation relative toeach other. In other embodiments, the ultrasound probes can besynchronized so that their relative positions are known or can beextracted. In some embodiments, optical markers 720 on multipleconventional ultrasound probes would allow registration of the multipleultrasound probe orientations in the same coordinate system.

In some further embodiments of the invention, other methods forassessing distance to tissues of interest, such as electricalconductivity, capacitance, or inductance of the tissues as mildelectrical current is applied.

In the modeling approach described above for visualizing soft tissues,it should be recognized that tracking a large number of landmarks helpsensure that the model is accurate. However, there is a trade-off thattracking a large number of landmarks may slow down the process, anddisallow real-time updating or require a lengthy registration process.In some embodiments, as fewer landmarks are tracked, tissue modeling topredict deformation of the non-tracked parts of the model becomesincreasingly important. In some embodiments, for tissue modeling, theelasticity and other mechanical qualities of the tissues are needed. Itmay be possible to assess the status of the tissues through a mechanismsuch as spectroscopy, where absorbance of light passed through tissuemight provide information on the composition of tissues, electricalconductivity, DEXA scan, MRI scan, CT scan or other means. Thisinformation could be provided to the computer model to allow betterestimation of soft tissue deformation.

Another possible mechanism for visualizing soft tissues can includeinjecting a conventional liquid tracer into the patient 18 that causesdifferent tissues to become temporarily detectable by an external scan.For example, the tracer could comprise a radioactive isotope that isattracted more to certain types of cells than others. Then, when thepatient is placed near an array of conventional radiation sensors, thesensors could detect the concentrations of the isotope in differentspatial locations.

Some embodiments include a mechanism to allow the user to control theadvancement and direction of a flexible catheter or wire (for examplewire 7405, 7410, 7600, or 8110) through an interface with the robot 15.In some embodiments, this mechanism can snap or lock into the robot'send-effectuator 30. In some embodiments, the guide tube 50 on therobot's end-effectuator 30 provides accurately controlled orientationand position of the catheter or wire at the point where it enters thepatient. In some embodiments, the mechanism would then allow the user tocontrol the rate and amount of advancement of the tube 50, the rate andamount of rotation of the tube 50, and activation of steering RF energy(for example, as described earlier with regard to steerable needle 7600in FIG. 73). In some embodiments, based on assumptions about thecondition of the soft tissues, and locations of obstacles such as bloodvessels, nerves, or organs between entry into the patient and thetarget, a non-linear path is planned by the software 3406 withparameters under the user's control. For example, in some embodiments,the method can include a command sequence such as “advance 5 mm,activate steering toward an azimuth of +35°, continue advancing 5 mmwhile rotating at 1° per second,” etc. In some embodiments, duringadvancement of the catheter or wire 7600 (or other wire 7405, 7410, or8110), the real-time location of the tip is tracked using LPS or othervisualization means. In some embodiments, the path plan is recalculatedbased on divergence from the expected path and advancement continues. Insome embodiments, this advancing/turning snap-in mechanism can also beused with beveled needles, taking advantage of the direction of thebevel, and the beveled face deflection force that moves the needlelaterally away from the face when advanced. In some embodiments,software 3406 would plan which direction the bevel should be orientedduring different phases of needle advancement.

In some embodiments, a mechanism similar to the one described above canalso be used for automatic hole preparation and insertion of screws. Forexample, in some embodiments, the end-effectuator 30 could have aconventional mechanism that would allow a tool to be retrieved from aconventional tool repository located somewhere outside the surgicalfield 17 In some embodiments, features on the tool holder would alloweasy automated engagement and disengagement of the tool. In someembodiments, after retrieving the tool, the end effectuator 30 wouldmove to the planned screw location and drill a pilot hole by rotatingthe assembly at an optimal drilling speed while advancing. In someembodiments, the system 1 would then guide the robot 15 to replace thedrill in the repository, and retrieve a driver with appropriately sizedscrew. In some embodiments, the screw would then be automaticallypositioned and inserted. In some embodiments, during insertion of thescrew, thrust and torque should be coordinated to provide good bite ofthe screw into bone. That is, the appropriate amount of forward thrustshould be applied during rotation so the screw will not strip the hole.

Some embodiments of the method also include algorithms for automaticallypositioning conventional screws. For example, in some embodiments,different considerations may dictate the decision of where the screwshould be placed. In some embodiments, it may be desirable to place thescrew into the bone such that the screw is surrounded by the thickest,strongest bone. In some embodiments, algorithms can be used to locatethe best quality bone from CT or DEXA scans, and to find an optimizedtrajectory such that the width of bone around the screw is thickest, orremains within cortical instead of cancellous bone for the greatestproportion. In some embodiments, it may be desirable to place the screwinto the bone at an entry point that is most perpendicular to the screw,or is at a “valley” instead of a peak or slope on the bonyarticulations. In some embodiments, by placing the screw in this way, itis less likely to skive or rotate during insertion and therefore likelyto end up in a more accurate inserted location. In some embodiments,algorithms can be used to assess the surface and find the best entrypoint to guide the screw to the target, while penetrating the boneperpendicular to the bone surface. In other embodiments, it may bedesirable to place screws in a multi-level case such that all the screwheads line up in a straight line or along a predictable curve. In someembodiments, by aligning screw heads in this way, the amount by whichthe surgeon must bend the interconnecting rod is minimized, reducing thetime of the procedure, and reducing weakening of the metal rod due torepeated bending. In some embodiments, algorithms can be used that keeptrack of anticipated head locations as they are planned, and suggestadjustments to trajectories that provide comparable bony purchase, butbetter rod alignment.

Some embodiments of the invention can use an LPS system that usestime-of-flight of RF signals from an emitter to an array of receivers tolocalize the position of the emitter. In some embodiments, it may bepossible to improve the accuracy of the LPS system by combining it withother modalities. For example, in some embodiments, it may be possibleuse a magnetic field, ultrasound scan, laser scan, CT, MRI or othermeans to assess the density and position of tissues and other media inthe region where the RF will travel. Since RF travels at different ratesthrough different media (air, tissue, metal, etc.), knowledge of thespatial orientation of the media through which the RF will travel willimprove the accuracy of the time-of-flight calculations.

In some embodiments, an enhancement to the robot 15 could includeinserting a conventional ultrasound probe into the guide tube 50. Insome embodiments, the ultrasound probe could be used as the guide tube50 penetrates through soft tissue to help visualize what is ahead. Asthe guide tube 50 advances, penetrating soft tissue and approachingbone, the ultrasound probe would be able to detect contours of the bonebeing approached. In some embodiments, this information could be used asa visual reference to verify that the actual anatomy being approached isthe same as the anatomy currently being shown on the 3D re-slicedmedical image over which the robot is navigating. For example, in someembodiments, if a small protrusion of bone is being approached deadcenter on the probe/guide tube 50 as it is pushed forward, the region inthe center of the ultrasound field representing the raised bone shouldshow a short distance to bone, while the regions toward the perimetershould show a longer distance to bone. In some embodiments, if theposition of the bony articulation on the re-sliced medical image doesnot appear to be lined up with the 2D ultrasound view of where the probeis approaching, this misalignment could be used to adjust theregistration of the robot 15 relative to the medical image. Similarly,in some embodiments, if the distance of the probe tip to bone does notmatch the distance perceived on the medical image, the registrationcould also be adjusted. In some embodiments, where the guide tube 50 isapproaching something other than bone, this method may also be usefulfor indicating when relative movement of internal soft tissues, organs,blood vessels, and nerves occurs.

Some embodiments can include a nerve sensing probe. For example, in someembodiments, for sensing whether a penetrating probe is near a nerve, anelectromyography (“EMG”) response to applied current could be used,enabling the ability of the robot 15 to steer around nerves. Forexample, as shown in FIG. 94, a probe 9400 could be used, with 1 or morecannulation offset from the probe's 9400 central axis that would enablea thin wire 9410 to extend from the tip 9405, ahead and to one side ofthe tip 9405. A beveled tip 9405 (or a conical or rounded tip) could beused.

In some embodiments, the probe 9400 could be advanced manually orautomatically and stopped, then the stimulating wire 9410 could beextended and current applied. In some embodiments, the EMG could bechecked to verify whether a nerve is in proximity. In some embodiments,the simulating wire 9410 could be retracted, and probe 9400 rotated sothat the portal for the stimulating wire 9410 is positioned at adifferent azimuth index. In some embodiments, the probe 9400 could againbe extended to check for the presence of nerves in a different regionahead. In some embodiments, if a nerve is encountered, it would be knownwhich direction the nerve is located, and which direction the probe 9400would need to be steered to avoid it. In some embodiments, instead of asingle wire 9410 extending and checking for a nerve, multiple wires 9410could simultaneously be extended from several portals around the probe9400. In some embodiments, the wires 9410 could be activated insequence, checking for EMG signals and identifying which wire 9410caused a response to identify the direction to avoid or steer. In someembodiments, it could be necessary to fully retract the stimulatingwires 9410 before attempting to further advance the probe 9400 to avoidblocking progress of the probe 9400. In some embodiments, thestimulating wires 9410 would have a small enough diameter so as to beable to penetrate a nerve without causing nerve damage.

As noted elsewhere in this application, the robot 15 executedtrajectories for paths into a patient 18 are planned using software (forexample, at least one module of the software 3406 running on thecomputing device 3401 including computer 100) where the desired vectorsare defined relative to radio opaque markers 730 on the image andtherefore relative to active markers 720 on the targeting fixture 690.In some embodiments, these trajectories can be planned at any time afterthe image is acquired, before or after registration is performed. Insome embodiments, it is possible that this trajectory planning can bedone on another computerized device. For example, in some embodiments, aconventional portable device (such as a tablet computer, or a laptopcomputer, or a smartphone computer) could be used. In some embodiments,the 3D image volume would be transferred to the portable device, and theuser would then plan and save the desired trajectories. In someembodiments, when robotic control is needed, this same image volumecould be loaded on the console that controls the robot 15 and thetrajectory plan could be transferred from the portable device. In someembodiments, using this algorithm, it would therefore be possible for aseries of patients 18 to each to have a targeting fixture 690 appliedand an imaging scan, such as a CT scan. In some embodiments, the 3Dvolume for each patient 18 could be exported to different portabledevices, and the same or different surgeons could plan trajectories foreach patient 18. In some embodiments, the same or different robot 15could then move from room to room. In some embodiments, in each room,the robot 15 would be sterilized (or have sterile draping applied, andwould receive the scan and trajectory plan. The robot 15 would thenexecute the plan, and then move to the next room to repeat the process.Similarly, the portion of the registration process in which the 3D imagevolume is searched for radio-opaque markers 730 could be performed onthe portable device. Then, in some embodiments, when the robot 15arrives, the registration information and the trajectories are bothtransferred to the robot 15 console. In some embodiments, by followingthis procedure, the time of computation of the image search algorithm onthe robot 15 console is eliminated, increasing efficiency of the overallprocess when the robot 15 is required in multiple rooms.

Although several embodiments of the invention have been disclosed in theforegoing specification, it is understood that many modifications andother embodiments of the invention will come to mind to which theinvention pertains, having the benefit of the teaching presented in theforegoing description and associated drawings. It is thus understoodthat the invention is not limited to the specific embodiments disclosedhereinabove, and that many modifications and other embodiments areintended to be included within the scope of the appended claims.Moreover, although specific terms are employed herein, as well as in theclaims which follow, they are used only in a generic and descriptivesense, and not for the purposes of limiting the described invention, northe claims which follow.

It will be appreciated by those skilled in the art that while theinvention has been described above in connection with particularembodiments and examples, the invention is not necessarily so limited,and that numerous other embodiments, examples, uses, modifications anddepartures from the embodiments, examples and uses are intended to beencompassed by the claims attached hereto. The entire disclosure of eachpatent and publication cited herein is incorporated by reference, as ifeach such patent or publication were individually incorporated byreference herein. Various features and advantages of the invention areset forth in the following claims.

What is claimed is:
 1. A surgical robot system comprising: a dynamicreference base attached to a patient fixture instrument, wherein thedynamic reference base has reflective markers, which are configured tobe tracked by a camera system, indicating a position of the patientfixture instrument in a camera coordinate system associated with thecamera system; and a calibration frame having reflective markers andradiopaque markers, the reflective markers configured to be trackable bythe camera system for indicating a location of a target anatomicalstructure in the camera coordinate system and the radiopaque markersindicating a location of the target anatomical structure in an imagecoordinate system defined by an imaging system, wherein the surgicalrobot system is configured to associate a spatial location of theradiopaque markers of the calibration frame in the image coordinatesystem to a spatial location of the reflective makers of the calibrationframe in the camera coordinate system for a registration of the targetanatomical structure in the camera coordinate system, wherein thesurgical robot system is configured to transfer the registration of thetarget anatomical structure to the dynamic reference base, wherein thedynamic reference base is configured to track the target anatomicalstructure in the camera coordinate system after registration occurs. 2.The surgical robot system of claim 1, wherein the calibration frame isremovably attached to the patient in proximity to the target anatomicalstructure.
 3. The surgical robot system of claim 2, wherein thecalibration frame is removed from the patient after registration of thetarget anatomical structure is transferred to the dynamic referencebase.
 4. The surgical robot system of claim 1, wherein the radio-opaquemarkers of the calibration frame can be located within a CT scan regionand reflective markers can be located outside of the CT scan region. 5.The surgical robot system of claim 1, wherein a surgical field can belocated within a perimeter created by radio-opaque markers.
 6. Thesurgical robot system of claim 1, wherein the calibration framecomprises an asymmetrical arrangement of the radiopaque markers of theregistration fixture device.
 7. The surgical robot system of claim 1,wherein three radiopaque markers are placed in an asymmetricalconfiguration on the calibration frame.
 8. The surgical robot system ofclaim 1, wherein a probe is used to register the coordinates of theradiopaque markers and the reflective markers.
 9. The surgical robotsystem of claim 1, wherein reflective markers are positioned on aportion of a robot arm in order to monitor a position of the robot armand the calibration frame or dynamic reference base simultaneously. 10.A surgical robot system comprising: a robot base having a display and anarm; a dynamic reference base attached to a patient fixture instrument,wherein the dynamic reference base has reflective markers, which areconfigured to be tracked by a camera system, indicating a position ofthe patient fixture instrument in a camera coordinate system associatedwith the camera system; and a calibration frame having reflectivemarkers and radiopaque markers, the reflective markers configured to betrackable by the camera system for indicating a location of a targetanatomical structure in the camera coordinate system and the radiopaquemarkers indicating a location of the target anatomical structure in animage coordinate system defined by an imaging system, wherein thesurgical robot system is configured to associate a spatial location ofthe radiopaque markers of the calibration frame in the image coordinatesystem to a spatial location of the reflective makers of the calibrationframe in the camera coordinate system for a registration of the targetanatomical structure in the camera coordinate system, wherein thesurgical robot system is configured to transfer the registration of thetarget anatomical structure to the coordinate system of the dynamicreference base, wherein the dynamic reference base is configured totrack the target anatomical structure in the camera coordinate systemafter registration occurs, wherein the robot arm is configured tooperated based on the coordinate system of the dynamic reference base.11. The surgical robot system of claim 12, wherein the calibration frameis removably attached to the patient.
 12. The surgical robot system ofclaim 11, wherein the calibration frame is removed from the patientafter registration of the target anatomical structure is transferred tothe dynamic reference base.
 13. The surgical robot system of claim 11,wherein the radio-opaque markers of the calibration frame can be locatedwithin a CT scan region and reflective markers can be located outside ofthe CT scan region.
 14. The surgical robot system of claim 11, wherein asurgical field can be located within a perimeter created by radio-opaquemarkers.
 15. The surgical robot system of claim 11, wherein thecalibration frame comprises an asymmetrical arrangement of theradiopaque markers of the registration fixture device.
 16. The surgicalrobot system of claim 11, wherein three radiopaque markers are placed inan asymmetrical configuration on the calibration frame.
 17. The surgicalrobot system of claim 11, wherein a probe is used to register thecoordinates of the radiopaque markers and the reflective markers. 18.The surgical robot system of claim 11, wherein reflective markers arepositioned on a portion of the robot arm in order to monitor a positionof the robot arm and the calibration frame or dynamic reference basesimultaneously.
 19. The surgical robot system of claim 11, wherein theradiopaque markers and reflective markers are sorted based on the rawcoordinates.
 20. A surgical robot system comprising: a robot base havinga display and an arm; a dynamic reference base attached to a patientfixture instrument, wherein the dynamic reference base has reflectivemarkers, which are configured to be tracked by a camera system,indicating a position of the patient fixture instrument in a cameracoordinate system associated with the camera system; a calibration framehaving reflective markers and radiopaque markers, the reflective markersconfigured to be trackable by the camera system for indicating alocation of a target anatomical structure in the camera coordinatesystem and the radiopaque markers indicating a location of the targetanatomical structure in an image coordinate system defined by an imagingsystem; and a probe embedded with reflective markers and in a knownrelationship with the tip of the probe, wherein the surgical robotsystem is configured to associate a spatial location of the radiopaquemarkers of the calibration frame in the image coordinate system to aspatial location of the reflective makers of the calibration frame inthe camera coordinate system for a registration of the target anatomicalstructure in the camera coordinate system; and wherein the surgicalrobot system is configured to transfer the registration of the targetanatomical structure to the coordinate system of the dynamic referencebase; wherein the dynamic reference base is configured to track thetarget anatomical structure in the camera coordinate system afterregistration occurs, wherein the probe is used to track the position ofthe radiopaque markers while recording the position of the probe tip andthe reflective markers on the frame simultaneously.